Conditional image repainting via semantic bridge and piecewise value function (unofficial)

Introducation

This is the author's unofficial PyTorch implementation.

We study Conditional Image Repainting (CIR) to lower the skill barriers of image editing techniques.

Conditional content generation refers to visual synthesis tasks conditioned on user inputs. The user inputs cover three aspects, i.e., geometry (semantic parsing mask), colors (attribute and language), and gray-scale textures (latent code).

By controlling the language and geomtery inputs, it is easy to generate objects with similar appearance or same shape for our model.

Our model support iterative image editing to modify the color, class or geometry of objects in the wild. After the editing, the whole scenes look quite different, which demonstrates the robustness and flexibility.

Prerequisites

• Python 3.6
• PyTorch 1.10
• NVIDIA GPU + CUDA cuDNN

Installation

Clone this repo:

git clone https://github.com/shuchenweng/TGC.git

Install PyTorch and dependencies from http://pytorch.org

Install other python requirements

Datasets

We process the CUB dataset for evaluation. For CUB, we annotate bird images with parsing masks, and follow AttnGAN for data processing.

Getting Started

Download the proccessed CUB dataset and copy them under DATA_DIR.

Download the pre-trained CUB weights and copy them under PRETRAINED_DIR.

Setting the MODEL_DIR as the storage directory for generated experimental results.

These directory parameters could be found in cfg/test_bird_SC.yml and cfg/train_bird_SC.yml.

1) Training

python main.py --cfg train_bird_SC.yml

2) Testing

python main.py --cfg test_bird_SC.yml

License

Licensed under a Creative Commons Attribution-NonCommercial 4.0 International.

Except where otherwise noted, this content is published under a CC BY-NC license, which means that you can copy, remix, transform and build upon the content as long as you do not use the material for commercial purposes and give appropriate credit and provide a link to the license.

Citation

If you use this code for your research, please cite our papers Conditional Image Repainting via Semantic Bridge and Piecewise Value Function

@inproceedings{Repaint,
  title={Conditional image repainting via semantic bridge and piecewise value function},
  author={Weng, Shuchen and Li, Wenbo and Li, Dawei and Jin, Hongxia and Shi, Boxin},
  booktitle={{ECCV}},
  year={2020},
}