Abstract:The "lifting from 2D pose" method has been the dominant approach to 3D Human Pose Estimation (3DHPE) due to the powerful visual analysis ability of 2D pose estimators. Widely known, there exists a depth ambiguity problem when estimating solely from 2D pose, where one 2D pose can be mapped to multiple 3D poses. Intuitively, the rich semantic and texture information in images can contribute to a more accurate "lifting" procedure. Yet, existing research encounters two primary challenges. Firstly, the distribution of image data in 3D motion capture datasets is too narrow because of the laboratorial environment, which leads to poor generalization ability of methods trained with image information. Secondly, effective strategies for leveraging image information are lacking. In this paper, we give new insight into the cause of poor generalization problems and the effectiveness of image features. Based on that, we propose an advanced framework. Specifically, the framework consists of two stages. First, we enable the keypoints to query and select the beneficial features from all image patches. To reduce the keypoints attention to inconsequential background features, we design a novel Pose-guided Transformer Layer, which adaptively limits the updates to unimportant image patches. Then, through a designed Adaptive Feature Selection Module, we prune less significant image patches from the feature map. In the second stage, we allow the keypoints to further emphasize the retained critical image features. This progressive learning approach prevents further training on insignificant image features. Experimental results show that our model achieves state-of-the-art performance on both the Human3.6M dataset and the MPI-INF-3DHP dataset.

Pose Estimation Via Complex-Frequency Domain Analysis of Image Gradient Orientations.

Partially Occluded Head Posture Estimation for 2D Images Using Pyramid HoG Features

A Study of Two Image Representations for Head Pose Estimation

Head Pose Estimation Using Covariance of Oriented Gradients

Head pose estimation from low-resolution image with hough forest

Estimating Head Pose Orientation Using Extremely Low Resolution Images

Head Pose Estimation Based on Detecting Facial Features

Head-Pose Estimation Based on Lateral Canthus Localizations in 2-D Images

Head Pose Estimation with Combined 2D SIFT and 3D HOG Features

Face pose estimation with combined 2D and 3D HOG features

Scale Robust Head Pose Estimation Based on Relative Homography Transformation

An Intuitive and Unconstrained 2D Cube Representation for Simultaneous Head Detection and Pose Estimation

2DHeadPose: A simple and effective annotation method for the head pose in RGB images and its dataset

Empirical Mode Decomposition-Based Facial Pose Estimation Inside Video Sequences

SD-Pose: facilitating space-decoupled human pose estimation via adaptive pose perception guidance

Head Pose Estimation Based on a Second Order Histogram of the Orientation Gradient

Orientation Cues-Aware Facial Relationship Representation for Head Pose Estimation via Transformer

Learning a Deep Regression Forest for Head Pose Estimation from a Single Depth Image

Human Pose Estimation in Monocular Omnidirectional Top-View Images

Lifting by Image -- Leveraging Image Cues for Accurate 3D Human Pose Estimation

Pose Estimating of Pose-Varied Face Image