Abstract:Human pose estimation heavily relies on the anatomical relationships among different body parts to locate keypoints. Despite the significant progress achieved by CNN-based models in human pose estimation, they typically fail to explicitly learn the global dependencies among various body parts. To overcome this limitation, we propose a spatial-aware human pose estimation model called SSpose that explicitly captures the spatial dependencies between specific key points and different locations in an image. The proposed SSpose model adopts a hybrid CNN-Transformer encoder to simultaneously capture local features and global dependencies. To better preserve image details, a multi-scale fusion module is introduced to integrate coarse and fine-grained image information. By establishing a connection with the activation maximization (AM) principle, the final attention layer of the Transformer aggregates contributions (i.e., attention scores) from all image positions and forms the maximum position in the heatmap, thereby achieving keypoint localization in the head structure. Additionally, to address the issue of visible information leakage in convolutional reconstruction, we have devised a self-supervised training framework for the SSpose model. This framework incorporates mask autoencoder (MAE) technology into SSpose models by utilizing masked convolution and hierarchical masking strategy, thereby facilitating efficient self-supervised learning. Extensive experiments demonstrate that SSpose performs exceptionally well in the pose estimation task. On the COCO val set, it achieves an AP and AR of 77.3% and 82.1%, respectively, while on the COCO test-dev set, the AP and AR are 76.4% and 81.5%. Moreover, the model exhibits strong generalization capabilities on MPII. Code is available at https://github.com/yulinfangylf/SSpose.

Learning Positional Priors for Pretraining 2D Pose Estimators.

Learning Heatmap-Style Jigsaw Puzzles Provides Good Pretraining for 2D Human Pose Estimation

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

STN-enhanced Message Passing Guided by Adversarial Learning for Human Pose Estimation

Prior-knowledge-based Self-Attention Network for 3D Human Pose Estimation

Learning Structure-Guided Diffusion Model for 2D Human Pose Estimation

AnatPose: Bidirectionally learning anatomy-aware heatmaps for human pose estimation

Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates

Multi-Person Pose Estimation in the Wild: Using Adversarial Method to Train a Top-Down Pose Estimation Network

Improving Occluded Human Pose Estimation Via Linked Joints

ProbPose: A Probabilistic Approach to 2D Human Pose Estimation

Heatmap Distribution Matching for Human Pose Estimation

Position Constrained Network for 3D Human Pose Estimation

Is 2D Heatmap Representation Even Necessary for Human Pose Estimation?

Denoising and Selecting Pseudo-Heatmaps for Semi-Supervised Human Pose Estimation

ActionPose: Pretraining 3D Human Pose Estimation with the Dark Knowledge of Action

Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning

SSpose: Self-supervised Spatial-aware Model for Human Pose Estimation

Hope: heatmap and offset for pose estimation

HEMlets PoSh: Learning Part-Centric Heatmap Triplets for 3D Human Pose and Shape Estimation

A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation