Abstract:The "lifting from 2D pose" method has been the dominant approach to 3D Human Pose Estimation (3DHPE) due to the powerful visual analysis ability of 2D pose estimators. Widely known, there exists a depth ambiguity problem when estimating solely from 2D pose, where one 2D pose can be mapped to multiple 3D poses. Intuitively, the rich semantic and texture information in images can contribute to a more accurate "lifting" procedure. Yet, existing research encounters two primary challenges. Firstly, the distribution of image data in 3D motion capture datasets is too narrow because of the laboratorial environment, which leads to poor generalization ability of methods trained with image information. Secondly, effective strategies for leveraging image information are lacking. In this paper, we give new insight into the cause of poor generalization problems and the effectiveness of image features. Based on that, we propose an advanced framework. Specifically, the framework consists of two stages. First, we enable the keypoints to query and select the beneficial features from all image patches. To reduce the keypoints attention to inconsequential background features, we design a novel Pose-guided Transformer Layer, which adaptively limits the updates to unimportant image patches. Then, through a designed Adaptive Feature Selection Module, we prune less significant image patches from the feature map. In the second stage, we allow the keypoints to further emphasize the retained critical image features. This progressive learning approach prevents further training on insignificant image features. Experimental results show that our model achieves state-of-the-art performance on both the Human3.6M dataset and the MPI-INF-3DHP dataset.

Body Structure Constraint for 3D Human Pose Estimation

Optimizing Network Structure for 3D Human Pose Estimation.

Double-chain Constraints for 3D Human Pose Estimation in Images and Videos

High-order local connection network for 3D human pose estimation based on GCN

Simplified-attention Enhanced Graph Convolutional Network for 3D human pose estimation

Semi-Dynamic Hypergraph Neural Network for 3D Pose Estimation

Regular Splitting Graph Network for 3D Human Pose Estimation

SPGformer: Serial–Parallel Hybrid GCN-Transformer With Graph-Oriented Encoder for 2-D-to-3-D Human Pose Estimation

Locally Connected Network for Monocular 3D Human Pose Estimation

Human Pose Estimation Based on Parallel Atrous Convolution and Body Structure Constraints

Conditional Directed Graph Convolution for 3D Human Pose Estimation

3D WholeBody Pose Estimation based on Semantic Graph Attention Network and Distance Information

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

A Deep Structure for Human Pose Estimation

3D Human Pose Estimation Using Improved Semantic Graph Convolutional Based on Fusing Non-local Neural Network and Multi-Head Attention

3D-UGCN: A Unified Graph Convolutional Network for Robust 3D Human Pose Estimation from Monocular RGB Images

A 3D Mesh-based Lifting-and-Projection Network for Human Pose Transfer

Relation-balanced graph convolutional network for 3D human pose estimation

Structure-aware human pose estimation with graph convolutional networks

Lifting by Image -- Leveraging Image Cues for Accurate 3D Human Pose Estimation

Interweaved Graph and Attention Network for 3D Human Pose Estimation