Abstract:The 6D pose estimation using RGBD images plays a pivotal role in robotics applications. At present, after obtaining the RGB and depth modality information, most methods directly concatenate them without considering information interactions. This leads to the low accuracy of 6D pose estimation in occlusion and illumination changes. To solve this problem, we propose a new method to fuse RGB and depth modality features. Our method effectively uses individual information contained within each RGBD image modality and fully integrates cross-modality interactive information. Specifically, we transform depth images into point clouds, applying the PointNet++ network to extract point cloud features; RGB image features are extracted by CNNs and attention mechanisms are added to obtain context information within the single modality; then, we propose a cross-modality feature fusion module (CFFM) to obtain the cross-modality information, and introduce a feature contribution weight training module (CWTM) to allocate the different contributions of the two modalities to the target task. Finally, the result of 6D object pose estimation is obtained by the final cross-modality fusion feature. By enabling information interactions within and between modalities, the integration of the two modalities is maximized. Furthermore, considering the contribution of each modality enhances the overall robustness of the model. Our experiments indicate that the accuracy rate of our method on the LineMOD dataset can reach 96.9%, on average, using the ADD (-S) metric, while on the YCB-Video dataset, it can reach 94.7% using the ADD-S AUC metric and 96.5% using the ADD-S score (<2 cm) metric.

Mitigating imbalances in heterogeneous feature fusion for multi-class 6D pose estimation

MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

MixedFusion: 6D Object Pose Estimation from Decoupled RGB-Depth Features.

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

An Efficient Color and Geometric Feature Fusion Module for 6D Object Pose Estiamtion

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

A Lightweight Color and Geometry Feature Extraction and Fusion Module for End-to-end 6D Pose Estimation

A Transformer-based multi-modal fusion network for 6D pose estimation

Homologous multimodal fusion network with geometric constraint keypoints selection for 6D pose estimation

Semi-Decoupled 6D Pose Estimation Via Multi-Modal Feature Fusion

An Iterative Attention Fusion Network for 6D Object Pose Estimation

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

FEIF: Feature Excitation and Interactive Fusion for 6D Object Pose Estimation.

Robust Classification and 6D Pose Estimation by Sensor Dual Fusion of Image and Point Cloud Data

6D Object Pose Estimation Based on Cross-Modality Feature Fusion

TemporalFusion: Temporal Motion Reasoning with Multi-Frame Fusion for 6D Object Pose Estimation

LHFF-Net: Local heterogeneous feature fusion network for 6DoF pose estimation

A Novel Depth and Color Feature Fusion Framework for 6D Object Pose Estimation.

A modal fusion network with dual attention mechanism for 6D pose estimation

DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion