Abstract:The growing demand for mobile devices has generated interest in lightweight human pose estimation. Currently, lightweight estimation generally uses heatmap-based methods, which has demonstrated exceptional performance. However, their use of non-differentiable post-processing imposes considerable inference latencies. Conversely, integral-based approaches expedite the inference process by employing a soft-argmax operation but compromise in accuracy. Integrating explicit heatmap knowledge learned using the heatmap-based method into the implicit heatmap generated by the integral-based method, thereby combining the best of both worlds, offers a promising avenue. However, owing to the disparities in supervision and inference processes, the explicit and implicit heatmaps are heterogeneous. Consequently, direct transfer of knowledge presents difficulties in ensuring consistencies in heat value and location. In this paper, we propose a novel Heterogeneous Heatmap Distillation (HHD) framework that effectively tackles these challenges. The framework seamlessly integrates explicit heatmap knowledge that contains high-precision localization information into implicit heatmaps. The framework revolves around an unbiased heatmap alignment scheme encompassing two steps: heterogeneous heatmap normalization and unbiased cropping. Heterogeneous heatmap normalization separately normalizes the output feature maps of both the teacher and student models, alleviating potential heat value bias during the knowledge transfer. Unbiased cropping applies closed-form computation on the normalized teacher and student heatmap to eliminate location bias. Additionally, mirror expansion is implemented to handle potential cases wherein the cropped region extends beyond the image boundary. Extensive experiments demonstrate the efficiency and effectiveness of our methods on the MSCOCO and MPII datasets compared to other integral-based lightweight networks. Our source codes and pre-trained models are available at https://github.com/ducongju/HHD .

SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

Context-Guided Adaptive Network for Efficient Human Pose Estimation.

Is 2D Heatmap Representation Even Necessary for Human Pose Estimation?

3D Human Pose Estimation Based on Wearable IMUs and Multiple Camera Views

PolarPose: Single-Stage Multi-Person Pose Estimation in Polar Coordinates

SD-Pose: facilitating space-decoupled human pose estimation via adaptive pose perception guidance

Parallel Self-Attention and Spatial-Attention Fusion for Human Pose Estimation and Running Movement Recognition

Composite Localization for Human Pose Estimation

PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds

Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention

Human Pose as Compositional Tokens

Lightweight Super-Resolution Head for Human Pose Estimation

LFSimCC: Spatial fusion lightweight network for human pose estimation

Heterogeneous heatmap distillation framework based on unbiased alignment for lightweight human pose estimation

Heatmap Distribution Matching for Human Pose Estimation

Center point to pose: Multiple views 3D human pose estimation for multi-person

FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions

An enhanced real-time human pose estimation method based on modified YOLOv8 framework

PoseDet: Fast Multi-Person Pose Estimation Using Pose Embedding

Optimized S2E Attention Block based Convolutional Network for Human Pose Estimation