ST3D

Code release for the paper ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection, CVPR 2021

Authors: Jihan Yang*, Shaoshuai Shi*, Zhe Wang, Hongsheng Li, Xiaojuan Qi (*equal contribution)

[arXiv];

News

[2022-09-26] ST3D++ (The extension of ST3D) has been accepted to T-PAMI. The new proposed modules are already integrated in this repo, the config will be released soon.

[2022-08-04] We are now supported DSNorm, SASD and HQAC in ST3D++. Besides, We support self-training for multi categories.

[2022-05-27] We are now supported OpenPCDet v0.3 with Spconv1.2 in branch st3d_v0.3.

Introduction

Our code is based on OpenPCDet v0.2. You can also checkout to the branch st3d_v0.3, which is based on OpenPCDet v0.3.
More updates on OpenPCDet are supposed to be compatible with our code.

Model Zoo

Waymo -> KITTI TASK

	method	Car@R11	Car@R40	download
SECOND-IoU	ST3D	63.53	62.19	model
PVRCNN	ST3D	62.76	64.05	model
SECOND-IoU	ST3D (w/ sn)	73.33	73.62	model
PVRCNN	ST3D (w/ sn)	75.71	77.33	model

We could not provide the above pretrained models due to Waymo Dataset License Agreement, but you should achieve similar performance by training with the default configs. To access these pretrained models, please send us an email with your name, institute, a screenshot of the the Waymo dataset registration confirmation mail, and your intended usage. Please send a second email if we don't get back to you in two days. Please note that Waymo open dataset is under strict non-commercial license so we are not allowed to share the model with you if it will used for any profit-oriented activities.

Also, the training Waymo data used in our work is version 1.0, but the version now available is version 1.2. The pretrained model on these two version data should be similar when adapted to KITTI.

nuScenes -> KITTI TASK

	method	Car@R11	Car@R40	download
SECOND-IoU	Source Only	22.55	17.92	model
SECOND-IoU	ROS	28.31	25.37	model
SECOND-IoU	SN	25.26	21.23	model
SECOND-IoU	ST3D	55.72	55.19	model
SECOND-IoU	ST3D (w/ SN)	62.55	62.27	model
PV-RCNN	Source Only	40.35	37.17	model
PV-RCNN	ROS	40.90	38.84	model
PV-RCNN	SN	50.62	49.47	model
PV-RCNN	ST3D	68.54	71.11	model
PV-RCNN	ST3D (w/ SN)	72.44	73.16	model

We provide all pretrained models here for nuScenes > KITTI task.

Waymo -> nuScenes TASK

	method	Car@R11	Car@R40	download
SECOND-IoU	ST3D	23.24	20.19	model
PVRCNN	ST3D	27.18	22.99	model
SECOND-IoU	ST3D (w/ sn)	23.52	20.38	model
PVRCNN	ST3D (w/ sn)	28.06	23.67	model

We could not provide the above pretrained models due to Waymo Dataset License Agreement, but you should achieve similar performance by training with the default configs.

Installation

Please refer to INSTALL.md for the installation.

Getting Started

Please refer to GETTING_STARTED.md to learn more usage about this project.

Supported features and ToDo List

• Support inference and pre-trained model

• Support training code on Waymo -> KITTI task

• Update to Latest OpenPCDet version.

• Support more adaptation tasks.

License

Our code is released under the Apache 2.0 license.

Acknowledgement

Our code is heavily based on OpenPCDet v0.2. Thanks OpenPCDet Development Team for their awesome codebase.

Citation

If you find this project useful in your research, please consider cite:

@inproceedings{yang2021st3d,
    title={ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection},
    author={Yang, Jihan and Shi, Shaoshuai and Wang, Zhe and Li, Hongsheng and Qi, Xiaojuan},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2021}
}

@article{yang2021st3d++,
  title={St3d++: Denoised self-training for unsupervised domain adaptation on 3d object detection},
  author={Yang, Jihan and Shi, Shaoshuai and Wang, Zhe and Li, Hongsheng and Qi, Xiaojuan},
  journal={arXiv preprint arXiv:2108.06682},
  year={2021}
}

@misc{openpcdet2020,
    title={OpenPCDet: An Open-source Toolbox for 3D Object Detection from Point Clouds},
    author={OpenPCDet Development Team},
    howpublished = {\url{https://github.com/open-mmlab/OpenPCDet}},
    year={2020}
}