This folder contains the training and evaluation code for CVA-MVSNet.
Create a conda environment and install requirements
conda create -n tandem python=3.7
conda activate tandem
conda install pytorch==1.5.1 torchvision==0.6.1 cudatoolkit=10.2 -c pytorch
pip install -r requirements.txt
Comment: The environment uses PyTorch 1.5.1 and PyTorch-Lightning 0.7.6 because this was the environment we used for development. However, we don't use features specific to this PyTorch version and thus an upgrade should hopefully be easy. On the other hand, PyTorch-Lightning introduced significant breaking changes and upgrading could be more cumbersome. Our model is defined in models/cva_mvsnet.py and wrapped by PyTorch-Lightning in models/tandem.py s.t. removing PyTorch-Lightning from the project should be possible.
Reproducibility: The requirements.txt does not contain version numbers for packages like numpy where we do not expect breaking changes. For reproducibility we include reproducibility/requirements_conda.txt and reproducibility/requirements_pip.txt with exact version numbers that were generated by pip freeze and conda list --export, respectively.
Config values are documented in config/default.yaml. In config/abl0*.yaml we include config files for the models shown in the ablation study (Table 2). You can start a training with
export TANDEM_DATA_DIR=/path/to/downloaded/tandem_replica
python train.py --config config/default.yaml path/to/out/folder DATA.ROOT_DIR $TANDEM_DATA_DIR
where path/to/out/folder may not exist and will contain tensorboard logs and checkpoints. You can overwrite more config parameters by listing them at the end of the command, e.g. TRAIN.EPOCHS 100.
For convenience we include pretrained models in pretrained/ablation and their evaluation numbers on TANDEM replica.
The pretrained models can be evaluated with
export TANDEM_DATA_DIR=/path/to/downloaded/tandem_replica
./eval.sh
Note: This will overwrite the *.txt and *.pkl files in pretrained/ablation.
We include the sbatch scripts we used on our SLURM cluster, but these might have to be adapted for a different setup.
We thank Xiaodong Gu, Zhiwen Fan, Zuozhuo Dai, Siyu Zhu, Feitong Tan, and Ping Tan for open sourcing their excellent work "Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching".