Abstract:Human pose estimation heavily relies on the anatomical relationships among different body parts to locate keypoints. Despite the significant progress achieved by CNN-based models in human pose estimation, they typically fail to explicitly learn the global dependencies among various body parts. To overcome this limitation, we propose a spatial-aware human pose estimation model called SSpose that explicitly captures the spatial dependencies between specific key points and different locations in an image. The proposed SSpose model adopts a hybrid CNN-Transformer encoder to simultaneously capture local features and global dependencies. To better preserve image details, a multi-scale fusion module is introduced to integrate coarse and fine-grained image information. By establishing a connection with the activation maximization (AM) principle, the final attention layer of the Transformer aggregates contributions (i.e., attention scores) from all image positions and forms the maximum position in the heatmap, thereby achieving keypoint localization in the head structure. Additionally, to address the issue of visible information leakage in convolutional reconstruction, we have devised a self-supervised training framework for the SSpose model. This framework incorporates mask autoencoder (MAE) technology into SSpose models by utilizing masked convolution and hierarchical masking strategy, thereby facilitating efficient self-supervised learning. Extensive experiments demonstrate that SSpose performs exceptionally well in the pose estimation task. On the COCO val set, it achieves an AP and AR of 77.3% and 82.1%, respectively, while on the COCO test-dev set, the AP and AR are 76.4% and 81.5%. Moreover, the model exhibits strong generalization capabilities on MPII. Code is available at https://github.com/yulinfangylf/SSpose.

Human Pose Estimation Via Multi-Resolution Convolutional Neural Network

Adaptively Fusing Complete Multi-resolution Features for Human Pose Estimation.

Multi-Scale Supervised Network for Human Pose Estimation

Modelling Human Body Pose for Action Recognition Using Deep Neural Networks

Human Pose Estimation Using Deep Structure Guided Learning.

3d Body Pose And Shape Estimation From Multi-View Images With Limb Geometric Constraint

Multi-Scale Structure-Aware Network for Human Pose Estimation

Human Pose Estimation Based on Parallel Atrous Convolution and Body Structure Constraints

Multi-Human Pose Estimation by Deep Learning-Based Sequential Approach for Human Keypoint Position and Human Body Detection

Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning

Full-Resolution Encoder-Decoder Networks with Multi-Scale Feature Fusion for Human Pose Estimation

3D Human Pose Estimation with Multi-Scale Graph Convolution and Hierarchical Body Pooling

Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention

Motion Capture for Sporting Events Based on Graph Convolutional Neural Networks and Single Target Pose Estimation Algorithms

Multi-Branch Network for Small Human Pose Estimation

Combining Local Appearance and Holistic View: Dual-Source Deep Neural Networks for Human Pose Estimation

Human Pose Estimation with Regression by Fusing Multi-View Visual Information

A Deconvolutional Bottom-up Deep Network for Multi-Person Pose Estimation.

Multi-Stage HRNet: Multiple Stage High-Resolution Network for Human Pose Estimation

Locally Connected Network for Monocular 3D Human Pose Estimation

SSpose: Self-supervised Spatial-aware Model for Human Pose Estimation