Unsupervised Homography Estimation with Coplanarity-Aware GAN

This is the Pytorch implementation of our CVPR2022 paper [PDF]

Code coming soon ...

Summary

Presentation Video:

[Bilibili] [Youtube]

Dependencies

pip install -r requirements.txt

Download the Deep Homography Dataset

Please refer to Content-Aware Unsupervised Deep Homography Estimation..

• Download raw dataset

# GoogleDriver
https://drive.google.com/file/d/19d2ylBUPcMQBb_MNBBGl9rCAS7SU-oGm/view?usp=sharing
# BaiduYun
https://pan.baidu.com/s/1Dkmz4MEzMtBx-T7nG0ORqA (key: gvor)

• Unzip the data to directory "./dataset"

• Run "video2img.py"

Be sure to scale the image to (640, 360) since the point coordinate system is based on the (640, 360).
e.g. img = cv2.imresize(img, (640, 360))

How to test?

python evaluate.py --model_dir ./experiments/HomoGAN/ --restore_file xxx.pth

How to train?

You need to modify ./dataset/data_loader.py slightly for your environment, and you can also refer to Content-Aware Unsupervised Deep Homography Estimation.

Pre-training:

1) set "pretrain_phase" in ./experiments/HomoGAN/params.json as True
2) python train.py --model_dir ./experiments/HomoGAN/

Fine-tuning:

1) set "pretrain_phase" in ./experiments/HomoGAN/params.json as False
2) python train.py --model_dir ./experiments/HomoGAN/ --restore_file xxx.pth

Citation

If you use this code or ideas from the paper for your research, please cite our paper:

@InProceedings{Hong_2022_CVPR,
    author    = {Hong, Mingbo and Lu, Yuhang and Ye, Nianjin and Lin, Chunyu and Zhao, Qijun and Liu, Shuaicheng},
    title     = {Unsupervised Homography Estimation With Coplanarity-Aware GAN},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {17663-17672}
}