Abstract:We present Ego3DPose, a highly accurate binocular egocentric 3D pose reconstruction system. The binocular egocentric setup offers practicality and usefulness in various applications, however, it remains largely under-explored. It has been suffering from low pose estimation accuracy due to viewing distortion, severe self-occlusion, and limited field-of-view of the joints in egocentric 2D images. Here, we notice that two important 3D cues, stereo correspondences, and perspective, contained in the egocentric binocular input are neglected. Current methods heavily rely on 2D image features, implicitly learning 3D information, which introduces biases towards commonly observed motions and leads to low overall accuracy. We observe that they not only fail in challenging occlusion cases but also in estimating visible joint positions. To address these challenges, we propose two novel approaches. First, we design a two-path network architecture with a path that estimates pose per limb independently with its binocular heatmaps. Without full-body information provided, it alleviates bias toward trained full-body distribution. Second, we leverage the egocentric view of body limbs, which exhibits strong perspective variance (e.g., a significantly large-size hand when it is close to the camera). We propose a new perspective-aware representation using trigonometry, enabling the network to estimate the 3D orientation of limbs. Finally, we develop an end-to-end pose reconstruction network that synergizes both techniques. Our comprehensive evaluations demonstrate that Ego3DPose outperforms state-of-the-art models by a pose estimation error (i.e., MPJPE) reduction of 23.1% in the UnrealEgo dataset. Our qualitative results highlight the superiority of our approach across a range of scenarios and challenges.

Data Snooping for Pose Estimation Based on Generalized Errors-in-Variables Model

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Unseen Object Pose Estimation via Registration

Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution.

Global Adaptation Meets Local Generalization: Unsupervised Domain Adaptation for 3D Human Pose Estimation.

Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation

The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation

SD-Pose: facilitating space-decoupled human pose estimation via adaptive pose perception guidance

Pose Estimation for Cross-Domain Non-Cooperative Spacecraft Based on Spatial-Aware Keypoints Regression

Seeing the Unseen: Wifi-based 2D Human Pose Estimation via an Evolving Attentive Spatial-Frequency Network

Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation

Pose Estimation Based On Gaussian Error Models

Semi-Supervised 2D Human Pose Estimation Driven by Position Inconsistency Pseudo Label Correction Module

Enhanced 3D Pose Estimation in Multi-Person, Multi-View Scenarios through Unsupervised Domain Adaptation with Dropout Discriminator

Head-Pose Estimation Based on Lateral Canthus Localizations in 2-D Images

Robust Estimation of 3D Human Poses from a Single Image

Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Images

Prior-information-guided corresponding point regression network for 6D pose estimation

Ego-Body Pose Estimation via Ego-Head Pose Estimation

Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views

A Simple Method to Boost Human Pose Estimation Accuracy by Correcting the Joint Regressor for the Human3.6m Dataset