RoMa: Revisiting Robust Losses for Dense Feature Matching

Project Page (TODO) | Paper

RoMa: Revisiting Robust Lossses for Dense Feature Matching
Johan Edstedt, Qiyu Sun, Georg Bökman, Mårten Wadenbäck, Michael Felsberg
Arxiv 2023

NOTE!!! Very early code, there might be bugs

The codebase is in the roma folder.

Setup/Install

In your python environment (tested on Linux python 3.10), run:

pip install -e .

Demo / How to Use

We provide two demos in the demos folder. Here's the gist of it:

from roma import roma_outdoor
roma_model = roma_outdoor(device=device)
# Match
warp, certainty = roma_model.match(imA_path, imB_path, device=device)
# Sample matches for estimation
matches, certainty = roma_model.sample(warp, certainty)
# Convert to pixel coordinates (RoMa produces matches in [-1,1]x[-1,1])
kptsA, kptsB = roma_model.to_pixel_coordinates(matches, H_A, W_A, H_B, W_B)
# Find a fundamental matrix (or anything else of interest)
F, mask = cv2.findFundamentalMat(
    kptsA.cpu().numpy(), kptsB.cpu().numpy(), ransacReprojThreshold=0.2, method=cv2.USAC_MAGSAC, confidence=0.999999, maxIters=10000
)

Reproducing Results

The experiments in the paper are provided in the experiments folder.

Training

1. First follow the instructions provided here: https://github.com/Parskatt/DKM for downloading and preprocessing datasets.
2. Run the relevant experiment, e.g.,

torchrun --nproc_per_node=4 --nnodes=1 --rdzv_backend=c10d experiments/roma_outdoor.py

Testing

python experiments/roma_outdoor.py --only_test --benchmark mega-1500

License

Due to our dependency on DINOv2, the license is sadly non-commercial only for the moment.

Acknowledgement

Our codebase builds on the code in DKM.

BibTeX

If you find our models useful, please consider citing our paper!

@article{edstedt2023roma,
title={{RoMa}: Revisiting Robust Lossses for Dense Feature Matching},
author={Edstedt, Johan and Sun, Qiyu and Bökman, Georg and Wadenbäck, Mårten and Felsberg, Michael},
journal={arXiv preprint arXiv:2305.15404},
year={2023}
}