Abstract:The 6D pose estimation using RGBD images plays a pivotal role in robotics applications. At present, after obtaining the RGB and depth modality information, most methods directly concatenate them without considering information interactions. This leads to the low accuracy of 6D pose estimation in occlusion and illumination changes. To solve this problem, we propose a new method to fuse RGB and depth modality features. Our method effectively uses individual information contained within each RGBD image modality and fully integrates cross-modality interactive information. Specifically, we transform depth images into point clouds, applying the PointNet++ network to extract point cloud features; RGB image features are extracted by CNNs and attention mechanisms are added to obtain context information within the single modality; then, we propose a cross-modality feature fusion module (CFFM) to obtain the cross-modality information, and introduce a feature contribution weight training module (CWTM) to allocate the different contributions of the two modalities to the target task. Finally, the result of 6D object pose estimation is obtained by the final cross-modality fusion feature. By enabling information interactions within and between modalities, the integration of the two modalities is maximized. Furthermore, considering the contribution of each modality enhances the overall robustness of the model. Our experiments indicate that the accuracy rate of our method on the LineMOD dataset can reach 96.9%, on average, using the ADD (-S) metric, while on the YCB-Video dataset, it can reach 94.7% using the ADD-S AUC metric and 96.5% using the ADD-S score (<2 cm) metric.

Robust Classification and 6D Pose Estimation by Sensor Dual Fusion of Image and Point Cloud Data

MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

A Transformer-based multi-modal fusion network for 6D pose estimation

An Efficient Color and Geometric Feature Fusion Module for 6D Object Pose Estiamtion

An Iterative Attention Fusion Network for 6D Object Pose Estimation

6D Object Pose Estimation Based on Cross-Modality Feature Fusion

FEIF: Feature Excitation and Interactive Fusion for 6D Object Pose Estimation.

MixedFusion: 6D Object Pose Estimation from Decoupled RGB-Depth Features.

Homologous multimodal fusion network with geometric constraint keypoints selection for 6D pose estimation

Semi-Decoupled 6D Pose Estimation Via Multi-Modal Feature Fusion

Mitigating imbalances in heterogeneous feature fusion for multi-class 6D pose estimation

A modal fusion network with dual attention mechanism for 6D pose estimation

A Pose Estimation Algorithm for Multimodal Data Fusion

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion

Towards Two-view 6D Object Pose Estimation: A Comparative Study on Fusion Strategy

A Lightweight Two-End Feature Fusion Network for Object 6D Pose Estimation

TemporalFusion: Temporal Motion Reasoning with Multi-Frame Fusion for 6D Object Pose Estimation