Abstract:We consider a category-level perception problem, where one is given 2D or 3D sensor data picturing an object of a given category (e.g., a car), and has to reconstruct the 3D pose and shape of the object despite intra-class variability (i.e., different car models have different shapes). We consider an active shape model, where -- for an object category -- we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape are estimated from 2D or 3D keypoints via non-convex optimization. Our first contribution is to develop PACE3D* and PACE2D*, the first certifiably optimal solvers for pose and shape estimation using 3D and 2D keypoints, respectively. Both solvers rely on the design of tight (i.e., exact) semidefinite relaxations. Our second contribution is to develop outlier-robust versions of both solvers, named PACE3D# and PACE2D#. Towards this goal, we propose ROBIN, a general graph-theoretic framework to prune outliers, which uses compatibility hypergraphs to model measurements' compatibility. We show that in category-level perception problems these hypergraphs can be built from the winding orders of the keypoints (in 2D) or their convex hulls (in 3D), and many outliers can be filtered out via maximum hyperclique computation. The last contribution is an extensive experimental evaluation. Besides providing an ablation study on simulated datasets and on the PASCAL3D+ dataset, we combine our solver with a deep keypoint detector, and show that PACE3D# improves over the state of the art in vehicle pose estimation in the ApolloScape datasets, and its runtime is compatible with practical applications. We release our code at <a class="link-external link-https" href="https://github.com/MIT-SPARK/PACE" rel="external noopener nofollow">this https URL</a>.

RANSAC Optimization for Category-level 6D Object Pose Estimation

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

RANSACs for 3D Rigid Registration: A Comparative Evaluation

CATRE: Iterative Point Clouds Alignment for Category-Level Object Pose Refinement

RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery

Globally Optimal Consensus Maximization for Relative Pose Estimation With Known Gravity Direction

ACR-Pose: Adversarial Canonical Representation Reconstruction Network for Category Level 6D Object Pose Estimation

A Method for Unseen Object Six Degrees of Freedom Pose Estimation Based on Segment Anything Model and Hybrid Distance Optimization

Fine segmentation and difference-aware shape adjustment for category-level 6DoF object pose estimation

Synthetic Depth Image-based Category-Level Object Pose Estimation with Effective Pose Decoupling and Shape Optimization

An Efficient Network for Category-Level 6D Object Pose Estimation

RNNPose: 6-DoF Object Pose Estimation Via Recurrent Correspondence Field Estimation and Pose Optimization

DR-Pose: A Two-stage Deformation-and-Registration Pipeline for Category-level 6D Object Pose Estimation

Toward Efficient and Robust Metrics for RANSAC Hypotheses and 3D Rigid Registration

SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variations

Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints

RANSAC Back to SOTA: A Two-stage Consensus Filtering for Real-time 3D Registration

RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization

Category-level Pose Estimation and Iterative Refinement for Monocular RGB-D Image

SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation

Robust 6-DoF Pose Estimation under Hybrid Constraints