This repository contains the code for the paper CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency.
Unsupervised domain adaptation algorithms aim to transfer the knowledge learned from one domain to another (e.g., synthetic to real images). The adapted representations often do not capture pixel-level domain shifts that are crucial for dense prediction tasks (e.g., semantic segmentation). In this paper, we present a novel pixel-wise adversarial domain adaptation algorithm. By leveraging image-to-image translation methods for data augmentation, our key insight is that while the translated images between domains may differ in styles, their predictions for the task should be consistent. We exploit this property and introduce a cross-domain consistency loss that enforces our adapted model to produce consistent predictions. Through extensive experimental results, we show that our method compares favorably against the state-of-the-art on a wide variety of unsupervised domain adaptation tasks.
If you find our code useful, please consider citing our work using the following bibtex:
@inproceedings{CrDoCo,
title={CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency},
author={Chen, Yun-Chun and Lin, Yen-Yu and Yang, Ming-Hsuan and Huang, Jia-Bin },
booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2019}
}
- Install Anaconda Python3.7
- This code is tested on NVIDIA RTX 2080 GPU with 24GB memory
- Please download the Cityscapes, GTA 5, NYUv2, and SUNCG datasets.
We prepare a demo code so that you can have a better understanding on the workflow of the code. Please refer to demo.py
We use the source code from Zhu et al.. Please follow the training tips here for pre-training the image-to-image translation networks. Please adjust the image size based on the GPU memory.
We found that the training of image-to-image translation network is not very stable since the domain gap between synthetic datasets and real-world datasets is large. Sometimes, the results will have severe visual artifacts or distortions. Based on our experience, we found that resizing images to resolution higher than 384x384 will implicitly alleviate this issue.
python train.py --model train --img_source_file /path/to/source/dataset --img_target_file /path/to/target/dataset --lab_source_file /path/to/source/label --lab_target_file /path/to/target/label --shuffle --flip --rotation
python test.py --model test --img_source_file /path/to/source/dataset --img_target_file /path/to/target/dataset
- This code is heavily borrowed from Zheng et al., Tsai et al., Zhu et al., Hoffman et al.