Abstract:6D pose recognition has been a crucial factor in the success of robotic grasping, and recent deep learning based approaches have achieved remarkable results on benchmarks. However, their generalization capabilities in real-world applications remain unclear. To overcome this gap, we introduce 6IMPOSE, a novel framework for sim-to-real data generation and 6D pose estimation. 6IMPOSE consists of four modules: First, a data generation pipeline that employs the 3D software suite Blender to create synthetic RGBD image datasets with 6D pose annotations. Second, an annotated RGBD dataset of five household objects was generated using the proposed pipeline. Third, a real-time two-stage 6D pose estimation approach that integrates the object detector YOLO-V4 and a streamlined, real-time version of the 6D pose estimation algorithm PVN3D optimized for time-sensitive robotics applications. Fourth, a codebase designed to facilitate the integration of the vision system into a robotic grasping experiment. Our approach demonstrates the efficient generation of large amounts of photo-realistic RGBD images and the successful transfer of the trained inference model to robotic grasping experiments, achieving an overall success rate of 87% in grasping five different household objects from cluttered backgrounds under varying lighting conditions. This is made possible by fine-tuning data generation and domain randomization techniques and optimizing the inference pipeline, overcoming the generalization and performance shortcomings of the original PVN3D algorithm. Finally, we make the code, synthetic dataset, and all the pre-trained models available on GitHub.

Open-vocabulary object 6D pose estimation

High-resolution open-vocabulary object 6D pose estimation

MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Open-Vocabulary Category-Level Object Pose and Size Estimation

OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation

OCSKB: An Object Component Sketch Knowledge Base for Fast 6D Pose Estimation

Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Tracking

Omni6D: Large-Vocabulary 3D Object Dataset for Category-Level 6D Object Pose Estimation

BOP: Benchmark for 6D Object Pose Estimation

POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference

ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers

6D object pose estimation via viewpoint relation reasoning

Object2Scene: Putting Objects in Context for Open-Vocabulary 3D Detection

Research on Model-Free 6D Object Pose Estimation Based on Vision 3D Matching.

BOP-Distrib: Revisiting 6D Pose Estimation Benchmark for Better Evaluation under Visual Ambiguities

ZeroPose: CAD-Prompted Zero-shot Object 6D Pose Estimation in Cluttered Scenes

From Words to Poses: Enhancing Novel Object Pose Estimation with Vision Language Models

6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping

CosyPose: Consistent multi-view multi-object 6D pose estimation

SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation