OpenCOOD is an Open COOperative Detection framework for autonomous driving. It is also the official implementation of the ICRA 2022 paper OPV2V.
02/20/2022: F-Cooper now is supported and the results/traiend model is in the benchmark page.
01/31/2022: Our paper OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication has been accpted by ICRA2022!
09/21/2021: OPV2V dataset is public available: https://mobility-lab.seas.ucla.edu/opv2v/
-
Provide easy data API for the Vehicle-to-Vehicle (V2V) multi-modal perception dataset OPV2V
It currently provides easy API to load LiDAR data from multiple agents simultaneously in a structured format and convert to PyTorch Tesnor directly for model use.
-
Provide multiple SOTA 3D detection backbone
It supports state-of-the-art LiDAR detector including PointPillar, Pixor, VoxelNet, and SECOND.
-
Support most common fusion strategies
It includes 3 most common fusion strategies: early fusion, late fusion, and intermediate fusion across different agents.
-
Support several SOTA multi-agent visual fusion model
It supports the most recent multi-agent perception algorithms (currently up to Sep. 2021) including Attentive Fusion, Cooper (early fusion), F-Cooper, etc. We will keep updating the newest algorithms.
-
Provide a convenient log replay toolbox for OPV2V dataset (coming soon)
It also provides an easy tool to replay the original OPV2V dataset. More importantly, it allows users to enrich the original dataset by attaching new sensors or define additional tasks (e.g. tracking, prediction) without changing the events in the initial dataset (e.g. positions and number of all vehicles, traffic speed).
Please refer to data introduction and installation guide to prepare data and install OpenCOOD. To see more details of OPV2V data, please check our website.
To quickly visualize the LiDAR stream in the OPV2V dataset, first modify the validate_dir
in your opencood/hypes_yaml/visualization.yaml
to the opv2v data path on your local machine, e.g. opv2v/validate
,
and the run the following commond:
cd ~/OpenCOOD
python opencood/visualization/vis_data_sequence.py
OpenCOOD uses yaml file to configure all the parameters for training. To train your own model from scratch or a continued checkpoint, run the following commonds:
python opencood/tools/train.py --hypes_yaml ${CONFIG_FILE} [--model_dir ${CHECKPOINT_FOLDER}]
Arguments Explanation:
hypes_yaml
: the path of the training configuration file, e.g.opencood/hypes_yaml/second_early_fusion.yaml
, meaning you want to train an early fusion model which utilizes SECOND as the backbone. See Tutorial 1: Config System to learn more about the rules of the yaml files.model_dir
(optional) : the path of the checkpoints. This is used to fine-tune the trained models. When themodel_dir
is given, the trainer will discard thehypes_yaml
and load theconfig.yaml
in the checkpoint folder.
Before you run the following command, first make sure the validation_dir
in config.yaml under your checkpoint folder
refers to the testing dataset path, e.g. opv2v_data_dumping/test
.
python opencood/tools/inference.py --model_dir ${CHECKPOINT_FOLDER} --fusion_method ${FUSION_STRATEGY} [--show_vis]
Arguments Explanation:
model_dir
: the path to your saved model.fusion_method
: indicate the fusion strategy, currently support 'early', 'late', and 'intermediate'.show_vis
: whether to visualize the detection overlay with point cloud.
The evaluation results will be dumped in the model directory.
We currently provide 3D LiDAR detection benchmark on OPV2V dataset, please refer to benchmark. Note that we keep updating the benchmark, so there are more methods there than in the paper. We suggest using the benchmark page when you try to compare with your own method in your research.
We have a series of tutorials to help you understand OpenCOOD more. Please check the series of our tutorials.
If you are using our OpenCOOD framework or OPV2V dataset for your research, please cite the following paper:
@inproceedings{xu2022opencood,
author = {Runsheng Xu, Hao Xiang, Xin Xia, Xu Han, Jinlong Li, Jiaqi Ma},
title = {OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication},
booktitle = {2022 IEEE International Conference on Robotics and Automation (ICRA)},
year = {2022}}
- Provide camera APIs for OPV2V
- Provide the log replay toolbox
- More SOTA models, such as DiscoNet and V2VNet.
OpenCOOD is supported by the UCLA Mobility Lab. We also appreciate the great work from OpenPCDet, as
part of our works use their framework.
- Dr. Jiaqi Ma (linkedin, UCLA Samueli)