SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

This is the official pytorch code for "SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal", which has been accepted by AAAI2022.

The training code, testing code, dataset, and pre-trained model have all been open sourced

Author

Zhaoyang Sun; Yaxiong Chen; Shengwu Xiong

News

• Our paper SHMT was accepted by NeurIPS2024. Paper link and code link.

• Our paper CSD-MT was accepted by CVPR2024. Paper link and code link.

• Our paper SSAT++ was accepted by TNNLS2023. Paper link and code link.

• Our paper SSAT was accepted by AAAI2022. Paper link and code link.

The framework of SSAT

Quick Start

1. Download the pre trained model and place it in the weights folder. Baidu Drive, password: 3dhd.
2. We have provided some examples, just run inference.py directly. python inference.py

Note that the quick_start folder is old and only contains test code.

How to test a custom dataset

1. Follow BeautyGAN to locate and crop facial images.
2. Prepare face parsing. Face parsing is used in this code. In our experiment, face parsing is generated by https://github.com/zllrunning/face-parsing.PyTorch.
3. Put the results of face parsing in the .\examples\seg1\makeup and \examples\seg1\non-makeup
4. python inference.py

Requirements

We recommend that you just use your own pytorch environment; the environment needed to run our model is very simple. If you do so, please ignore the following environment creation.

A suitable conda environment named SSAT can be created and activated with:

conda env create -f environment.yaml
conda activate SSAT

Download our dataset

Our dataset can be downloaded here Baidu Drive, password: cdrb.

Extract the downloaded file and place it on top of this folder.

Training code

We have set the default hyperparameters in the options.py file, please modify them yourself if necessary. To train the model, please run the following command directly

python train.py

Inference code

python inference.py

Our results

Citation

If this work is helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{sun2022ssat,
  title={Ssat: A symmetric semantic-aware transformer network for makeup transfer and removal},
  author={Sun, Zhaoyang and Chen, Yaxiong and Xiong, Shengwu},
  booktitle={Proceedings of the AAAI Conference on artificial intelligence},
  pages={2325--2334},
  year={2022}
}

Acknowledgement

Some of the codes are build upon PSGAN, Face Parsing and aster.Pytorch.

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.