Abstract:The task of object pose estimation in computer vision heavily relies on both color (RGB) and depth (D) images to provide crucial appearance and geometric information, assisting algorithms in understanding occlusions and object geometry, thereby enhancing accuracy. However, the dependency on specialized sensors capable of capturing depth poses challenges in terms of cost and availability. Consequently, researchers are exploring methods to estimate object poses solely from RGB images. Nevertheless, this approach encounters difficulties in handling occlusions, discerning object geometry, and resolving ambiguities arising from similar color or texture patterns. This paper introduces a novel geometry-aware method for object pose estimation utilizing RGB images as input to determine the poses of multiple object instances. Our approach leverages both depth and color images during training but only relies on color images during inference. Departing from traditional depth sensors, our method computes predicted point clouds directly from estimated depth images derived from RGB inputs. A key innovation lies in the formulation of a multi-scale fusion module adept at seamlessly integrating features extracted from RGB images with those inferred from the predicted point clouds. This fusion process significantly fortifies the pose estimation pipeline by harnessing the strengths of both modalities, resulting in notably improved object poses. Extensive experimentation demonstrates that our approach markedly outperforms state-of-the-art RGB-based methods on Occluded-LINEMOD and YCB-Video datasets. Moreover, our method achieves competitive results compared to RGB-D approaches that necessitate both RGB and depth data from physical sensors.

An RGB-D Based Approach for Human Pose Estimation

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Zero-Shot 3d Pose Estimation of Unseen Object by Two-Step Rgb-D Fusion

Towards Robust RGB-D Human Mesh Recovery

Fast Human Detection in RGB-D Images Based on Color-Depth Joint Feature Learning.

A Survey of Efficient Regression of General-Activity Human Poses from Depth Images

Detecting and tracking people in real time with RGB-D camera

Accurate Estimation of Human Body Orientation from Rgb-D Sensors

RDPN6D: Residual-based Dense Point-wise Network for 6Dof Object Pose Estimation Based on RGB-D Images

Object Pose Estimation Using Color Images and Predicted Depth Maps

Robust 6DoF Pose Estimation Against Depth Noise and a Comprehensive Evaluation on a Mobile Dataset

Robust Estimation of 3D Human Poses from a Single Image

Depth-based Privileged Information for Boosting 3D Human Pose Estimation on RGB

Learning to Estimate Object Poses Without Real Image Annotations

A Survey on Monocular 3D Human Pose Estimation

CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-Dof Object Pose Estimation

Multi-Person 3D Pose Estimation from Multi-View Uncalibrated Depth Cameras

3D Human Pose Estimation = 2D Pose Estimation + Matching

Real-time RGBD-based Extended Body Pose Estimation

Single Image Depth Map Estimation for Improving Posture Recognition

Monocular 3D Human Pose Estimation by Predicting Depth on Joints