Abstract:The growing demand for mobile devices has generated interest in lightweight human pose estimation. Currently, lightweight estimation generally uses heatmap-based methods, which has demonstrated exceptional performance. However, their use of non-differentiable post-processing imposes considerable inference latencies. Conversely, integral-based approaches expedite the inference process by employing a soft-argmax operation but compromise in accuracy. Integrating explicit heatmap knowledge learned using the heatmap-based method into the implicit heatmap generated by the integral-based method, thereby combining the best of both worlds, offers a promising avenue. However, owing to the disparities in supervision and inference processes, the explicit and implicit heatmaps are heterogeneous. Consequently, direct transfer of knowledge presents difficulties in ensuring consistencies in heat value and location. In this paper, we propose a novel Heterogeneous Heatmap Distillation (HHD) framework that effectively tackles these challenges. The framework seamlessly integrates explicit heatmap knowledge that contains high-precision localization information into implicit heatmaps. The framework revolves around an unbiased heatmap alignment scheme encompassing two steps: heterogeneous heatmap normalization and unbiased cropping. Heterogeneous heatmap normalization separately normalizes the output feature maps of both the teacher and student models, alleviating potential heat value bias during the knowledge transfer. Unbiased cropping applies closed-form computation on the normalized teacher and student heatmap to eliminate location bias. Additionally, mirror expansion is implemented to handle potential cases wherein the cropped region extends beyond the image boundary. Extensive experiments demonstrate the efficiency and effectiveness of our methods on the MSCOCO and MPII datasets compared to other integral-based lightweight networks. Our source codes and pre-trained models are available at https://github.com/ducongju/HHD .

A lightweight attention-driven distillation model for human pose estimation

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

Lightweight and Effective Human Pose Estimation Model Based on Multi-Angle Knowledge Distillation

Lightweight Human Pose Estimation Using Loss Weighted by Target Heatmap.

Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention

Efficient and Lightweight Human Pose Estimation Algorithm

Efficient Pose Estimation Via a Lightweight Single-Branch Pose Distillation Network

Simple and Lightweight Human Pose Estimation

Online Knowledge Distillation for Efficient Pose Estimation

Knowledge Distillation for Lightweight 2D Single-Person Pose Estimation

Integral Knowledge Distillation for Multi-Person Pose Estimation

Effective Whole-body Pose Estimation with Two-stages Distillation

A Lightweight Context-Aware Feature Transformer Network for Human Pose Estimation

Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation

Shift Pose: A Lightweight Transformer-like Neural Network for Human Pose Estimation

Lightweight Human Pose Estimation Based on Adaptive Feature Sensing

Heterogeneous heatmap distillation framework based on unbiased alignment for lightweight human pose estimation

DistilPose: Tokenized Pose Regression with Heatmap Distillation

Low-resolution human pose estimation

Attention-Enhanced Lightweight Hourglass Network for Human Pose Estimation