Skip to content

Latest commit

 

History

History
620 lines (530 loc) · 67.3 KB

README.md

File metadata and controls

620 lines (530 loc) · 67.3 KB

Note: For any missing or recently published papers, feel free to pull a request, we will add them asap :)

Introduction

This is the official repository of ''Deep Learning-Based Object Pose Estimation: A Comprehensive Survey''. Specifically, we first introduce the datasets used for object pose estimation. Then, we review the instance-level, category-level, and unseen methods, respectively. Finally, we summarize the common applications of this task. The taxonomy of this survey is shown as follows

A comparison of instance-level, category-level, and unseen methods is shown as follows. Instance-level methods can only estimate the pose of specific object instances on which they are trained. Category-level methods can infer intra-class unseen instances rather than being limited to specific instances in the training data. In contrast, unseen object pose estimation methods have stronger generalization ability and can handle object categories not encountered during training.

Datasets

Chronological overview of the datasets for object pose estimation evaluation. Notably, the pink arrows represent the BOP Challenge datasets, which can be used to evaluate both instance-level and unseen object methods. The red references represent the datasets of articulated objects.

2.1 Datasets for Instance-Level Methods

All Datasets

2.2 Datasets for Category-Level Methods

2.2.1 Rigid Objects Datasets
2.2.2 Articulated Objects Datasets

2.3 Datasets for Unseen Methods

All Datasets

Instance-Level

Instance-level object pose estimation describes the task of estimating the pose of the objects that have been seen during the training of the model. We classify existing instance-level methods into four categories: correspondence-based, template-based, voting-based, and regression-based methods. The illustration of these four categories is shown as follows

3.1 Correspondence-Based Methods

3.1.1 Sparse Correspondence Methods

2017

  • BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects Without Using Depth [Paper]
  • 6-dof Object Pose from Semantic Keypoints [Paper] [Code]
  • Robust 3D Object Tracking from Monocular Images Using Stable Parts [Paper]

2018

  • Real-Time Seamless Single Shot 6D Object Pose Prediction [Paper]
  • Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation [Paper]

2019

  • Segmentation-Driven 6D Object Pose Estimation [Paper]
  • Learning Local RGB-to-CAD Correspondences for Object Pose Estimation [Paper]

2020

  • HOPE-Net: A Graph-Based Model for Hand-Object Pose Estimation [Paper]
  • HybridPose: 6D Object Pose Estimation Under Hybrid Representations [Paper]
  • Learning Deep Network for Detecting 3D Object Keypoints and 6D Poses [Paper]
  • Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images [Paper]

2021

  • MFPN-6D : Real-time One-stage Pose Estimation of Objects on RGB Images [Paper]
  • Wide-Depth-Range 6D Object Pose Estimation in Space [Paper]
  • Confidence-Based 6D Object Pose Estimation [Paper]
  • DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-Scale Consistency [Paper]
  • Semi-Supervised 3D Hand-Object Poses Estimation With Interactions in Time [Paper] [Code]
  • Keypoint-Graph-Driven Learning Framework for Object Pose Estimation [Paper]
  • PyraPose: Feature Pyramids for Fast and Accurate Object Pose Estimation under Domain Shift [Paper]
  • GhostPose: Multi-view Pose Estimation of Transparent Objects for Robot Hand Grasping [Paper]

2023

  • Knowledge Distillation for 6D Pose Estimation by Aligning Distributions of Local Predictions [Paper]
  • Linear-Covariance Loss for End-to-End Learning of 6D Pose Estimation [Paper]
  • CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Network [Paper] [Code]
3.1.2 Dense Correspondence Methods

2019

  • CDPN: Coordinates-based Disentangled Pose Network for Real-time RGB-Based 6-DoF Object Pose Estimation [Paper] [Code]
  • DPOD: 6D Pose Object Detector and Refiner [Paper] [Code]
  • Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation [Paper] [Code]

2020

  • Reconstruct Locally, Localize Globally: A Model Free Method for Object Pose Estimation [Paper]
  • EPOS: Estimating 6D Pose of Objects with Symmetries [Paper] [Code]

2021

  • GeoPose: Dense Reconstruction Guided 6D Object Pose Estimation With Geometric Consistency [Paper]
  • DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation [Paper]
  • Pseudo-Siamese Graph Matching Network for Textureless Objects’ 6-D Pose Estimation [Paper] [Code]
  • SD-Pose: Semantic Decomposition for Cross-Domain 6D Object Pose Estimation [Paper]

2022

  • ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation [Paper] [Code]
  • EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation [Paper] [Code]
  • BiCo-Net: Regress Globally, Match Locally for Robust 6D Pose Estimation [Paper] [Code]
  • Neural Correspondence Field for Object Pose Estimation [Paper] [Code]
  • Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimation [Paper] [Code]
  • SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings [Paper] [Code]
  • Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [Paper] [Code]

2023

  • Center-Based Decoupled Point Cloud Registration for 6D Object Pose Estimation [Paper] [Code]
  • Geometric-aware dense matching network for 6D pose estimation of objects from RGB-D images [Paper] [Code]
  • NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation [Paper]
  • Pseudo Flow Consistency for Self-Supervised 6D Object Pose Estimation [Paper] [Code]

2024

  • HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimation [Paper] [Code]
  • RNNPose: 6-DoF Object Pose Estimation Via Recurrent Correspondence Field Estimation and Pose Optimization [Paper] [Code]

3.2 Template-Based Methods

3.2.1 RGB-Based Template Methods

2018

  • Implicit 3D Orientation Learning for 6D Object Detection from RGB Images [Paper] [Code]

2019

  • 6DoF Pose Estimation with Object Cutout Based on a Deep Autoencoder [Paper]

2020

  • 6D Object Pose Estimation Algorithm Using Preprocessing of Segmentation and Keypoint Extraction [Paper]
  • Domain-Translated 3D Object Pose Estimation [Paper]
  • Pose-guided Auto-Encoder and Feature-Based Refinement for 6-DoF Object Pose Regression [Paper]
  • Spatial Attention Improves Iterative 6D Object Pose Estimation [Paper] [Code]

2021

  • PoseRBPF: A Rao–Blackwellized Particle Filter for 6-D Object Pose Tracking [Paper] [Code]
3.2.2 Point Cloud-Based Template Methods

2022

  • DCL-Net: Deep Correspondence Learning Network for 6D Pose Estimation [Paper] [Code]

2023

  • SE(3) Diffusion Model-based Point Cloud Registration for Robust 6D Object Pose Estimation [Paper] [Code]

2024

  • Match Normalization: Learning-Based Point Cloud Registration for 6D Object Pose Estimation in the Real World [Paper]

3.3 Voting-Based Methods

3.3.1 Indirect Voting Methods

2019

  • PVNet: Pixel-Wise Voting Network for 6DoF Pose Estimation [Paper] [Code]

2020

  • PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation [Paper] [Code]
  • 6DoF Object Pose Estimation via Differentiable Proxy Voting Loss [Paper]

2021

  • KDFNet: Learning Keypoint Distance Field for 6D Object Pose Estimation [Paper]
  • FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation [Paper] [Code]
  • ParametricNet: 6DoF Pose Estimation Network for Parametric Shapes in Stacked Scenarios [Paper] [Code]

2022

  • DGECN: A Depth-Guided Edge Convolutional Network for End-to-End 6D Pose Estimation [Paper]
  • Spatial Feature Mapping for 6DoF Object Pose Estimation [Paper]
  • Vote from the Center: 6 DoF Pose Estimation in RGB-D Images by Radial Keypoint Voting [Paper] [Code]
  • MV6D: Multi-View 6D Pose Estimation on RGB-D Frames Using a Deep Point-wise Voting Network [Paper]
  • Learning to Estimate Object Poses without Real Image Annotations [Paper] [Code]
  • Sim2Real Instance-Level Style Transfer for 6D Pose Estimation [Paper]

2023

  • BDR6D: Bidirectional Deep Residual Fusion Network for 6D Pose Estimation [Paper]
  • KVNet: An iterative 3D keypoints voting network for real-time 6-DoF object pose estimation [Paper]
  • Deep Fusion Transformer Network with Weighted Vector-Wise Keypoints Voting for Robust 6D Object Pose Estimation [Paper] [Code]

2024

  • 6D-Diff: A Keypoint Diffusion Framework for 6D Object Pose Estimation [Paper]
3.3.2 Direct Voting Methods

2019

  • DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion [Paper] [Code]

2020

  • A Novel Depth and Color Feature Fusion Framework for 6D Object Pose Estimation [Paper]
  • PPR-Net++: Accurate 6-D Pose Estimation in Stacked Scenarios [Paper] [Code]

2021

  • Robust 6D Object Pose Estimation by Learning RGB-D Features [Paper] [Code]
  • PR-GCN: A Deep Graph Convolutional Network With Point Refinement for 6D Pose Estimation [Paper] [Code]
  • TemporalFusion: Temporal Motion Reasoning with Multi-Frame Fusion for 6D Object Pose Estimation [Paper] [Code]

2022

  • ES6D: A Computation Efficient and Symmetry-Aware 6D Pose Regression Framework [Paper] [Code]
  • SC6D: Symmetry-agnostic and Correspondence-free 6D Object Pose Estimation [Paper] [Code]

2023

  • A Depth Adaptive Feature Extraction and Dense Prediction Network for 6-D Pose Estimation in Robotic Grasping [Paper]

3.4 Regression-Based Methods

3.4.1 Geometry-Guided Regression Methods

2020

  • 6D Object Pose Regression via Supervised Learning on Point Clouds [Paper]
  • G2L-Net: Global to Local Network for Real-Time 6D Pose Estimation With Embedding Vector Features [Paper] [Code]
  • Single-Stage 6D Object Pose Estimation [Paper]

2021

  • 6D Object Pose Estimation with Pairwise Compatible Geometric Features [Paper]
  • CloudAAE: Learning 6D Object Pose Regression with On-Line Data Synthesis on Point Clouds [Paper] [Code]
  • Semi-Supervised 6D Object Pose Estimation Without Using Real Annotations [Paper]
  • StablePose: Learning 6D Object Poses From Geometrically Stable Patches [Paper] [Code]
  • GCCN: Geometric Constraint Co-attention Network for 6D Object Pose Estimation [Paper]
  • SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation [Paper] [Code]
  • GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation [Paper] [Code]
  • Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation [Paper] [Code]

2022

  • Trans6D: Transformer-Based 6D Object Pose Estimation and Refinement [Paper]

2023

  • SMOC-Net: Leveraging Camera Pose for Self-Supervised Monocular Object Pose Estimation [Paper]
  • NVR-Net: Normal Vector Guided Regression Network for Disentangled 6D Pose Estimation [Paper]

2024

  • PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking [Paper]
  • HFT6D: Multimodal 6D Object Pose Estimation Based on Hierarchical Feature Transformer [Paper]
3.4.2 Direct Regression Methods

2017

  • SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again [Paper] [Code]
  • Deep 6-DOF Tracking [Paper] [Code]
  • PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes [Paper] [Code]

2018

  • Learning 6DoF Object Poses from Synthetic Single Channel Images [Paper]
  • Real-Time Object Pose Estimation with Pose Interpreter Networks [Paper] [Code]
  • Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image [Paper] [Code]
  • A Unified Framework for Multi-View Multi-Class Object Pose Estimation [Paper]
  • DeepIM: Deep Iterative Matching for 6D Pose Estimation [Paper] [Code]
  • Deep Model-Based 6D Pose Refinement in RGB [Paper] [Code]
  • Object Detection and Pose Estimation Based on Convolutional Neural Networks Trained with Synthetic Data [Paper]

2019

  • PCRNet: Point Cloud Registration Network using PointNet Encoding [Paper] [Code]
  • Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data [Paper]
  • 3D Object Pose Estimation Using Multi-Objective Quaternion Learning [Paper]
  • Regression-Based Three-Dimensional Pose Estimation for Texture-Less Objects [Paper] [Code]

2020

  • Single Shot 6D Object Pose Estimation [Paper]
  • EfficientPose: An Efficient, Accurate and Scalable End-to-End 6D Multi Object Pose Estimation Approach [Paper] [Code]
  • Self6D: Self-Supervised Monocular 6D Object Pose Estimation [Paper] [Code]
  • CosyPose: Consistent Multi-view Multi-object 6D Pose Estimation [Paper] [Code]
  • se(3)-TrackNet: Data-Driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains [Paper] [Code]

2021

  • Pose Estimation from RGB Images of Highly Symmetric Objects using a Novel Multi-Pose Loss and Differential Rendering [Paper] [Code]
  • VIPose: Real-Time Visual-Inertial 6D Object Pose Tracking [[Paper]](VIPose: Real-time Visual-Inertial 6D Object Pose Tracking)
  • RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering [Paper] [Code]

2022

  • DProST: Dynamic Projective Spatial Transformer Network for 6D Pose Estimation [Paper] [Code]
  • Uni6D: A Unified CNN Framework Without Projection Breakdown for 6D Pose Estimation [Paper]
  • HFF6D: Hierarchical Feature Fusion Network for Robust 6D Object Pose Tracking [Paper]

2023

  • Shape-Constraint Recurrent Flow for 6D Object Pose Estimation [Paper] [Code]

2024

  • Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3) [Paper]
  • MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlation [Paper] [Code]

Category-Level

Research on category-level methods has garnered significant attention due to their potential for generalizing to unseen objects within established categories. We classify them into two categories: shape prior-based and shape prior-free methods. The illustration of these two categories is shown as follows

4.1 Shape Prior-Based Methods

4.1.1 NOCS Shape Alignment Methods

2020

  • Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation [Paper] [Code]

2021

  • Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks [Paper]
  • SGPA: Structure-Guided Prior Adaptation for Category-Level 6D Object Pose Estimation [Paper] [Code]
  • ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation [Paper]

2022

  • 6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-Based Instance Representation Learning [Paper] [Code]
  • Towards Self-Supervised Category-Level Object Pose and Size Estimation [Paper]
  • Object Level Depth Reconstruction for Category Level 6D Object Pose Estimation from Monocular RGB Image [Paper]

2023

  • Category-Level 6D Pose Estimation Using Geometry-Guided Instance-Aware Prior and Multi-Stage Reconstruction [Paper]
  • StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS [Paper] [Code]
  • DR-Pose: A Two-Stage Deformation-and-Registration Pipeline for Category-Level 6D Object Pose Estimation [Paper] [Code]
  • GPT-COPE: A Graph-Guided Point Transformer for Category-Level Object Pose Estimation [Paper]
  • SD-Pose: Structural Discrepancy Aware Category-Level 6D Object Pose Estimation [Paper]
  • Generative Category-Level Shape and Pose Estimation with Semantic Primitives [Paper] [Code]
  • RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery [Paper] [Code]

2024

  • CatFormer: Category-Level 6D Object Pose Estimation with Transformer [Paper] [Code]
4.1.2 Direct Regress Pose Methods

2022

  • SAR-Net: Shape Alignment and Recovery Network for Category-Level 6D Object Pose and Size Estimation [Paper] [Code]
  • CenterSnap: Single-Shot Multi-Object 3D Shape Reconstruction and Categorical 6D Pose and Size Estimation [Project Page] [Paper] [Code]
  • ShAPO: Implicit Representations for Multi-Object Shape Appearance and Pose Optimization [Project Page] [Paper] [Code]
  • SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation [Paper]
  • RBP-Pose: Residual Bounding Box Projection for Category-Level Pose Estimation [Paper] [Code]
  • CATRE: Iterative Point Clouds Alignment for Category-Level Object Pose Refinement [Paper] [Code]
  • Category-Level 6D Object Pose and Size Estimation Using Self-supervised Deep Prior Deformation Networks [Paper] [Code]
  • Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset [Paper] [Code]

2023

  • Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild [Paper] [Code]
  • i2c-net: Using Instance-Level Neural Networks for Monocular Category-Level 6D Pose Estimation [Paper]
  • DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field [Paper]

2024

  • FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects [Project Page] [Paper] [Video]
  • DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation [Project Page] [Paper] [Code]
  • MH6D: Multi-Hypothesis Consistency Learning for Category-Level 6-D Object Pose Estimation [Paper] [Code]
  • GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement [Paper] [Code]

4.2 Shape Prior-Free Methods

4.2.1 Depth-Guided Geometry-Aware Methods

2020

  • Category-Level Articulated Object Pose Estimation [Paper] [Code]

2021

  • FS-Net: Fast Shape-Based Network for Category-Level 6D Object Pose Estimation With Decoupled Rotation Mechanism [Paper] [Code]
  • CAPTRA: CAtegory-Level Pose Tracking for Rigid and Articulated Objects From Point Clouds [Paper] [Code]
  • GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion [Paper] [Code]
  • Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation from Point Clouds [Paper] [Code]

2022

  • GPV-Pose: Category-Level Object Pose Estimation via Geometry-Guided Point-Wise Voting [Paper] [Code]
  • CPPF: Towards Robust Category-Level 9D Pose Estimation in the Wild [Paper] [Code]
  • iCaps: Iterative Category-Level Object Pose and Shape Estimation [Paper]
  • Category-Level 6D Object Pose Estimation With Structure Encoder and Reasoning Attention [Paper]

2023

  • Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance [Paper] [Code]
  • HS-Pose: Hybrid Scope Feature Extraction for Category-Level Object Pose Estimation [Paper] [Code]
  • Query6DoF: Learning Sparse Queries as Implicit Shape Prior for Category-Level 6DoF Pose Estimation [Paper] [Code]
  • SOCS: Semantically-Aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations [Paper] [Code]
  • Generative Category-level Object Pose Estimation via Diffusion Models [Paper] [Code]
  • Category-Level Articulated Object 9D Pose Estimation via Reinforcement Learning [Paper]

2024

  • Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation [Paper] [Code]
  • Category-Level Multi-Part Multi-Joint 3D Shape Assembly [Paper]
4.2.2 RGBD-Guided Semantic and Geometry Fusion Methods

2019

  • Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation [Paper] [Code]

2020

  • Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation [Paper]
  • 6-PACK: Category-Level 6D Pose Tracker with Anchor-Based Keypoints [Paper] [Code]

2021

  • DualPoseNet: Category-Level 6D Object Pose and Size Estimation Using Dual Pose Network With Refined Learning of Pose Consistency [Paper] [Code]
  • BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models [Paper] [Code]
  • Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space [Paper] [Code]

2022

  • Self-Supervised Category-Level 6D Object Pose Estimation with Deep Implicit Shape Representation [Paper] [Code]
  • UDA-COPE: Unsupervised Domain Adaptation for Category-Level Object Pose Estimation [Paper]
  • Attention-guided RGB-D Fusion Network for Category-level 6D Object Pose Estimation [Paper]

2023

  • TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation [Paper]
  • IST-Net: Prior-Free Category-Level Pose Estimation with Implicit Space Transformation [Paper] [Code]
  • VI-Net: Boosting Category-Level 6D Object Pose Estimation via Learning Decoupled Rotations on the Spherical Representations [Paper] [Code]

2024

  • SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation [Paper] [Code]
4.2.3 Others

2020

  • Category Level Object Pose Estimation via Neural Analysis-by-Synthesis [Paper] [Code]

2021

  • Category-Level Metric Scale Object Shape and Pose Estimation [Paper]
  • iNeRF: Inverting Neural Radiance Fields for Pose Estimation [Paper]

2022

  • Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image [Paper]
  • Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation [Paper]
  • A Visual Navigation Perspective for Category-Level Object Pose Estimation [Paper] [Code]
  • Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features [Paper]
  • TransNet: Category-Level Transparent Object Pose Estimation [Paper]
  • Zero-Shot Category-Level Object Pose Estimation [Paper] [Code]

2023

  • CatTrack: Single-Stage Category-Level 6D Object Pose Tracking via Convolution and Vision Transformer [Paper]
  • Self-Supervised Category-Level 6D Object Pose Estimation With Optical Flow Consistency [Paper]

2024

  • OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation [Paper] [Code]
  • Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [Paper]
  • CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledge [Paper]

Unseen

Unseen object pose estimation methods can generalize to unseen objects without the need for retraining. We classify existing unseen methods into two categories: CAD model-based and manual reference view-based methods. The illustration of these two categories is shown as follows

5.1 CAD Model-Based Methods

5.1.1 Feature Matching-Based Methods

2019

  • CorNet: Generic 3D Corners for 6D Pose Estimation of New Objects without Retraining [Paper]

2020

  • 3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings [Paper]

2022

  • Unseen Object 6D Pose Estimation: A Benchmark and Baselines [Paper] [Code]

2023

  • Learning Symmetry-Aware Geometry Correspondences for 6D Object Pose Estimation [Paper] [Code]
  • KeyMatchNet: Zero-Shot Pose Estimation in 3D Point Clouds by Generalized Keypoint Matching [Paper]
  • ZeroPose: CAD-Model-based Zero-Shot Pose Estimation [Paper]

2024

  • FreeZe: Training-Free Zero-Shot 6D Pose Estimation with Geometric and Vision Foundation Models [Paper] [Code]
  • MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images [Paper]
  • SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation [Paper] [Code]
5.1.2 Template Matching-Based Methods

2015-2019

  • Learning Descriptors for Object Recognition and 3D Pose Estimation [Paper]
  • Pose Guided RGBD Feature Learning for 3D Object Pose Estimation [Paper]
  • Pose Estimation for Objects with Rotational Symmetry [Paper] [Code]

2020

  • Multi-path Learning for Object Pose Estimation Across Domains [Paper] [Code]

2021

2022

  • I Like to Move It: 6D Pose Estimation as an Action Decision Process [Paper]
  • DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation [Paper] [Code]
  • Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions [Paper] [Code]
  • Fusing Local Similarities for Retrieval-based 3D Orientation Estimation of Unseen Objects [Paper] [Code]
  • Self-Supervised Vision Transformers for 3D Pose Estimation of Novel Objects [Paper]
  • OVE6D: Object Viewpoint Encoding for Depth-Based 6D Object Pose Estimation [Paper] [Code]
  • OSOP: A Multi-Stage One Shot Object Pose Estimation Framework [Paper]
  • MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [Paper] [Code]

2023

  • Diff-DOPE: Differentiable Deep Object Pose Estimation [Paper] [Code]
  • ZS6D: Zero-Shot 6D Object Pose Estimation using Vision Transformers [Paper] [Code]
  • FoundPose: Unseen Object Pose Estimation with Foundation Features [Paper] [Code]

2024

  • FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]
  • Object Pose Estimation via the Aggregation of Diffusion Features [Paper] [Code]
  • GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence [Paper] [Code]
  • GenFlow: Generalizable Recurrent Flow for 6D Pose Refinement of Novel Objects [Paper]

5.2 Manual Reference View-Based Methods

5.2.1 Feature Matching-Based Methods

2021

  • Unseen Object Pose Estimation via Registration [Paper]

2022

  • FS6D: Few-Shot 6D Pose Estimation of Novel Objects [Paper] [Code]
  • OnePose: One-Shot Object Pose Estimation without CAD Models [Paper] [Code]
  • OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models [Paper] [Code]

2023

  • POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference [Paper] [Code]
  • PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching [Paper] [Code]

2024

  • Open-Vocabulary Object 6D Pose Estimation [Paper] [Code]
  • MFOS: Model-Free & One-Shot Object Pose Estimation [Paper]
5.2.2 Template Matching-Based Methods

2020

  • LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation[Paper] [Code]

2022

  • PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking [Paper] [Code]
  • Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images [Paper] [Code]

2023

  • SA6D: Self-Adaptive Few-Shot 6D Pose Estimator for Novel and Occluded Objects [Paper]
  • BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects [Paper] [Code]

2024

  • NOPE: Novel Object Pose Estimation from a Single Image [Paper] [Code]
  • LocPoseNet: Robust Location Prior for Unseen Object Pose Estimation [Paper] [Code]
  • Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images [Paper] [Code]
  • GS-Pose: Cascaded Framework for Generalizable Segmentation-based 6D Object Pose Estimation [Paper] [Code]
  • FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]

Applications

Chronological overview of some representative applications of object pose estimation methods. The black references, red references, and orange references represent the application of instance-level, category-level, and unseen methods, respectively.

6.1 Robotic Manipulation

6.1.1 Instance-Level Manipulation
  • Weakly supervised 6D pose estimation for robotic grasping [Paper]
  • Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects [Paper] [Code]
  • Detect in RGB, Optimize in Edge: Accurate 6D Pose Estimation for Texture-less Industrial Parts [Paper]
  • PPR-Net:Point-wise Pose Regression Network for Instance Segmentation and 6D Pose Estimation in Bin-picking Scenarios [Paper]
  • MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion [Paper] [Code]
  • GhostPose: Multi-view Pose Estimation of Transparent Objects for Robot Hand Grasping [Paper]
  • Tomato harvesting robotic system based on Deep-ToMaToS: Deep learning network using transformation loss for 6D pose estimation of maturity classified tomatoes with side-stem [Paper]
  • 6D Robotic Assembly Based on RGB-only Object Pose Estimation [Paper]
  • Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Picking [Paper]
  • AttentionVote: A coarse-to-fine voting network of anchor-free 6D pose estimation on point cloud for robotic bin-picking application [Paper]
6.1.2 Category-Level Manipulation
  • Robotic Continuous Grasping System by Shape Transformer-Guided Multiobject Category-Level 6-D Pose Estimation [Paper] [Code]
  • Fine Segmentation and Difference-Aware Shape Adjustment for Category-Level 6DoF Object Pose Estimation [Paper]
  • Category-Level 6-D Object Pose Estimation With Shape Deformation for Robotic Grasp Detection [Paper]
  • ICK-Track: A Category-Level 6-DoF Pose Tracker Using Inter-Frame Consistent Keypoints for Aerial Manipulation [Paper] [Code]
  • Robotic Grasp Detection Based on Category-Level Object Pose Estimation With Self-Supervised Learning [Paper]
  • Domain-Generalized Robotic Picking via Contrastive Learning-Based 6-D Pose Estimation [Paper]
6.1.3 Unseen Object Manipulation
  • ZePHyR: Zero-shot Pose Hypothesis Rating [Paper] [Code]
  • MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare [Paper] [Code]
  • FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]

6.2 Augmented Reality/Virtual Reality

All Methods
  • Deep Multi-state Object Pose Estimation for Augmented Reality Assembly [Paper]
  • Efficient 6-DoF Tracking of Handheld Objects from an Egocentric Viewpoint [Paper]
  • Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images [Paper] [Code]
  • OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models [Paper] [Code]
  • FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects [Paper] [Code]

6.3 Aerospace

All Methods
  • Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering [Paper]
  • Wide-Depth-Range 6D Object Pose Estimation in Space [Paper] [Code]
  • CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Space [Paper] [Code]
  • 6D Object Pose Estimation from Approximate 3D Models for Orbital Robotics [Paper]

6.4 Hand-Object Interaction

All Methods
  • Object Learning for 6D Pose Estimation and Grasping from RGB-D Videos of In-hand Manipulation [Paper]
  • Harmonious Feature Learning for Interactive Hand-Object Pose Estimation [Paper] [Code]
  • Hierarchical Graph Neural Networks for Proprioceptive 6D Pose Estimation of In-hand Objects [Paper]
  • HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields [Paper] [Code]

6.5 Autonomous Driving

All Methods
  • Deep Learning for 6D Pose Estimation of Objects — A Case Study for Autonomous Driving [Paper]
  • PanelPose: A 6D Pose Estimation of Highly-Variable Panel Object for Robotic Robust Cockpit Panel Inspection [Paper]

Citation

If you find the paper useful, please cite our paper.

@article{liu2024survey,
  title={Deep Learning-Based Object Pose Estimation: A Comprehensive Survey},
  author={Liu, Jian and Sun, Wei and Yang, Hui and Zeng, Zhiwen and Liu, Chongpei and Zheng, Jin and Liu, Xingyu and Rahmani, Hossein and Sebe, Nicu and Mian, Ajmal},  
  journal={arXiv preprint arXiv:2405.07801},
  year={2024}
}

Contact

Due to the one-sided nature of our knowledge, if you find any issues or have any suggestions, please feel free to post an issue or contact us via email