CBMNet(CVPR 2023, highlight)

Official repository for the CVPR 2023 paper, "Event-based Video Frame Interpolation with Cross-Modal Asymmetric Bidirectional Motion Fields"

[Paper] [Supp] [Oral(YouTube)]

Qualitative video demos on ERF-X170FPS dataset

Falling pop-corn

Flowers

Driving scene

ERF-X170FPS dataset

Dataset of high-resolution (1440x975), high-fps (170fps) video frames plus high resolution events with extremely large motion using the beam-splitter acquisition system:

Quantitative results on the ERF-X170FPS datasets

Downloading ERF-X170FPS datasets

You can download the raw-data(collected frame and events) from this links

• [RAW-Train]
• [RAW-Test]

** Cautions: the x,y coordinates of the raw event file are multiplied by 128.

Requirements

• PyTorch 1.8.0
• CUDA 11.2
• python 3.8

Quick Usage

Download repository:

    $ git clone https://github.com/intelpro/CBMNet

Install correlation package:

    $ sh install_correlation.sh

Download network weights(trained on ERF-X170FPS datasets) and place downloaded model in ./pretrained_model/

• [Ours]
• [Ours-Large]

Generate an intermediate video frame using ours model:

    $ python run_samples.py  --model_name ours --ckpt_path pretrained_model/ours_weight.pth --save_output_dir ./output --image_number 0

Also, you can generate intermediate video frame using ours-large model:

    $ python run_samples.py  --model_name ours_large --ckpt_path pretrained_model/ours_large_weight.pth --save_output_dir ./output --image_number 0

Pretrained model

The model pretrained on the BS-ERGB dataset can be downloaded from the following link:

• [Ours-Large(BS-ERGB)]

Reference

Taewoo Kim, Yujeong Chae, Hyun-kyurl Jang, and Kuk-Jin Yoon" Event-based Video Frame Interpolation with Cross-modal Asymmetric Bidirectional Motion Fields", In CVPR, 2023.

@InProceedings{Kim_2023_CVPR,
    author    = {Kim, Taewoo and Chae, Yujeong and Jang, Hyun-Kurl and Yoon, Kuk-Jin},
    title     = {Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fields},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {18032-18042}
}

Contact

If you have any question, please send an email to taewoo([email protected])

License

The project codes and datasets can be used for research and education only.