Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data

Official pytorch implementation of paper "Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data". (CVPR 2022)

🚀 Run model through Replicate's web demo here:

Demo and Docker image on Replicate

Quick Usage

Download the pretrained model from here, and put it in the "ckpt" directory. Then run the following script:

python easy_use.py

As default, the input images are expected to be put in "data" directory and the results will be saved in "results" directory. You can also change them by different arguments.

Synthetic Dataset

Download the synthetic dataset in Google Drive.

The meaning of filename: img-[Glass]-[Subject]-[Expression]-[Node Type]-[HDR]-[HDR Rotation]-[Image Type]

Glass: Glass ID
Subject: Subject ID
Expression: Expression Name
Node Type: 4 kinds of floating nodes on different positions of the nose. {0, 1, 2, 3}
HDR: HDR Lighting Name
HDR Rotation: The angle of vertical axis in degree
Image Type: Images with different visibility combinations of eyeglasses and shadows or segmentations

To get the label of shadow segmentation:

python generate_shadow_label.py --syndata_dir <synthetic_data_dir>

Citation

If our paper helps your research, please cite it in your publications:

@inproceedings{lyu2022portrait,
  title={Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data},
  author={Lyu, Junfeng and Wang, Zhibo and Xu, Feng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={3429--3439},
  year={2022}
}