Abstract:The growing demand for mobile devices has generated interest in lightweight human pose estimation. Currently, lightweight estimation generally uses heatmap-based methods, which has demonstrated exceptional performance. However, their use of non-differentiable post-processing imposes considerable inference latencies. Conversely, integral-based approaches expedite the inference process by employing a soft-argmax operation but compromise in accuracy. Integrating explicit heatmap knowledge learned using the heatmap-based method into the implicit heatmap generated by the integral-based method, thereby combining the best of both worlds, offers a promising avenue. However, owing to the disparities in supervision and inference processes, the explicit and implicit heatmaps are heterogeneous. Consequently, direct transfer of knowledge presents difficulties in ensuring consistencies in heat value and location. In this paper, we propose a novel Heterogeneous Heatmap Distillation (HHD) framework that effectively tackles these challenges. The framework seamlessly integrates explicit heatmap knowledge that contains high-precision localization information into implicit heatmaps. The framework revolves around an unbiased heatmap alignment scheme encompassing two steps: heterogeneous heatmap normalization and unbiased cropping. Heterogeneous heatmap normalization separately normalizes the output feature maps of both the teacher and student models, alleviating potential heat value bias during the knowledge transfer. Unbiased cropping applies closed-form computation on the normalized teacher and student heatmap to eliminate location bias. Additionally, mirror expansion is implemented to handle potential cases wherein the cropped region extends beyond the image boundary. Extensive experiments demonstrate the efficiency and effectiveness of our methods on the MSCOCO and MPII datasets compared to other integral-based lightweight networks. Our source codes and pre-trained models are available at https://github.com/ducongju/HHD .

Scale-aware heatmap representation for human pose estimation

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

Multi-Person Pose Estimation with Accurate Heatmap Regression and Greedy Association

Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention

Heatmap Distribution Matching for Human Pose Estimation

Multi-Scale Structure-Aware Network for Human Pose Estimation

Distribution-Aware Coordinate Representation for Human Pose Estimation

Is 2D Heatmap Representation Even Necessary for Human Pose Estimation?

Low-resolution human pose estimation

Multi-Scale Supervised Network for Human Pose Estimation

Multi-person pose estimation using atrous convolution

Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates

Multi-Stage HRNet: Multiple Stage High-Resolution Network for Human Pose Estimation

Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation

Denoising and Selecting Pseudo-Heatmaps for Semi-Supervised Human Pose Estimation

Hybrid model for Single-Stage Multi-Person Pose Estimation

Lightweight Super-Resolution Head for Human Pose Estimation

Improving Robustness for Pose Estimation Via Stable Heatmap Regression

A Deconvolutional Bottom-up Deep Network for Multi-Person Pose Estimation.

Heterogeneous heatmap distillation framework based on unbiased alignment for lightweight human pose estimation

Learning Heatmap-Style Jigsaw Puzzles Provides Good Pretraining for 2D Human Pose Estimation