Author: Rajat Talak
We evaluate various methods of estimating object pose, given partial object point cloud and the object CAD model. We compare various methods in the literature, namely, TEASER++, DeepGMR, PointNetLK, EquiPose, and several semantic keypoint-based methods used in C-3PO. We would like to thank the Authors of these repos, as we use their implementation and training setup, to generate results.
Clone the repository:
git clone --recurse-submodules --depth 1 https://github.com/MIT-SPARK/pose-baselines.git
cd pose-baselines/Install conda environment:
conda create --name pb-viz python=3.9
conda activate pb-viz
conda install scipy
conda install pip
pip install numpy
pip install pandas
pip install seaborn
pip install notebook
pip install ipywidgets
pip install pickle5Make sure you are in the results directory (i.e. pose-baselines/results). Run jupyter notebook:
conda activate pb-viz
jupyter notebook results.ipynbYou should see the following:
Create two empty folders named data_shapenet and data_ycb in:
- pose-baselines/deepgmr/
- pose-baselines/EquiPose/equi-pose/
- pose-baselines/fpfh_teaser/
- pose-baselines/PointNetLK_Revisited/
Our experiments rely on the ShapeNet, KeypointNet, and the YCB datasets. Please view ShapeNet's terms of use here. There's no need to download the datasets seperately. Our experiments use a processed version of these datasets. Follow the steps below to download and save the relevant dataset.
Download our processed dataset files on Google Drive here.
We've provided the dataset as a zip archive split into 1GB chunks of the format c3po-data.z**.
Combine and unzip the folder:
zip -F c3po-data.zip --out data.zip
unzip data.zipVerify your directory structure looks as follows:
data
|
|─── learning-objects
│
│─── KeypointNet
│
└─── ycb
Move the KeypointNet and learning-objects folders in data_shapenet and ycb folder in data_ycb. You may even create symlink.
Install conda environment:
cd fpfh_teaser
conda env create -f environment.yml
conda activate pb-fpfhInstall TEASER++ by following the instructions in TEASER++. Clone the TEASER++ repo in a new directory:
git clone https://github.com/MIT-SPARK/TEASER-plusplus.git Install TEASER++ in fpfh conda environment as follows:
sudo apt install cmake libeigen3-dev libboost-all-dev
conda activate fpfh
cd TEASER-plusplus
cd TEASER-plusplus && mkdir build && cd build
cmake -DTEASERPP_PYTHON_VERSION=3.8 .. && make teaserpp_python
cd python && pip install .This installs teaserpp_python.
Run evaluations:
cd fpfh_teaser
conda activate pb-fpfh
bash scripts/test.shnote: uncomment out "from pytorch3d import ops" in pose-baselines/utils_common.py
Install conda environment:
cd deepgmr
conda env create -f environment.yml
conda activate pb-deepgmr
conda install -c open3d-admin -c conda-forge open3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"
pip install pickle5
pip install seabornRun evaluations:
cd deepgmr
conda activate pb-deepgmr
bash scripts/test.shnote: uncomment out "from pytorch3d import ops" in pose-baselines/utils_common.py
Install conda environment:
cd PointNetLK_Revisited
conda create --name pb-lk python=3.8
conda activate pb-lk
conda install pip
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
pip install -r requirements.txt
pip install tensorboard
pip install pickle5
pip install seabornRun evaluations:
cd PointNetLK_Revisited
conda activate pb-lk
bash scripts/test.shnote: either comment out "from pytorch3d import ops" in pose-baselines/utils_common.py or install pytorch3d in pb-lk.
Install conda environment:
cd EquiPose/equi-pose/
conda create --name pb-equi-pose python=3.8
conda activate pb-equi-pose
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
pip install -r new_requirements.txt
bash build.sh
pip install tensorboardor install the environment as specified in pose-baselines/EquiPose/equi=pose/README.md.
Download the pre-trained models drive_link provided by EquiPose. Add them to the folder: pose-baselines/EquiPose/logs/model/
Run evaluations:
cd EquiPose/equi-pose/
conda activate pb-equi-pose
bash scripts/test.shnote: either comment out "from pytorch3d import ops" in pose-baselines/utils_common.py or install pytorch3d in pb-equi-pose.
If you find this project useful, do cite our work:
@article{Talak23arxiv-c3po,
title = {Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training},
author = {Talak, Rajat and Peng, Lisa and Carlone, Luca},
year = {2023},
month = {Jan.},
journal = {arXiv preprint arXiv: 2206.11215},
eprint = {2206.11215},
note = {\linkToPdf{https://arxiv.org/pdf/2206.11215.pdf}},
pdf={https://arxiv.org/pdf/2206.11215.pdf},
Year = {2023}
}
This project is released under MIT license.
This work was partially funded by ARL DCIST CRA W911NF-17-2-0181, ONR RAIDER N00014-18-1-2828, and NSF CAREER award "Certifiable Perception for Autonomous Cyber-Physical Systems".