Abstract:The growing demand for mobile devices has generated interest in lightweight human pose estimation. Currently, lightweight estimation generally uses heatmap-based methods, which has demonstrated exceptional performance. However, their use of non-differentiable post-processing imposes considerable inference latencies. Conversely, integral-based approaches expedite the inference process by employing a soft-argmax operation but compromise in accuracy. Integrating explicit heatmap knowledge learned using the heatmap-based method into the implicit heatmap generated by the integral-based method, thereby combining the best of both worlds, offers a promising avenue. However, owing to the disparities in supervision and inference processes, the explicit and implicit heatmaps are heterogeneous. Consequently, direct transfer of knowledge presents difficulties in ensuring consistencies in heat value and location. In this paper, we propose a novel Heterogeneous Heatmap Distillation (HHD) framework that effectively tackles these challenges. The framework seamlessly integrates explicit heatmap knowledge that contains high-precision localization information into implicit heatmaps. The framework revolves around an unbiased heatmap alignment scheme encompassing two steps: heterogeneous heatmap normalization and unbiased cropping. Heterogeneous heatmap normalization separately normalizes the output feature maps of both the teacher and student models, alleviating potential heat value bias during the knowledge transfer. Unbiased cropping applies closed-form computation on the normalized teacher and student heatmap to eliminate location bias. Additionally, mirror expansion is implemented to handle potential cases wherein the cropped region extends beyond the image boundary. Extensive experiments demonstrate the efficiency and effectiveness of our methods on the MSCOCO and MPII datasets compared to other integral-based lightweight networks. Our source codes and pre-trained models are available at https://github.com/ducongju/HHD .

Deviation Wing Loss for High-Performance 2D Pose Estimation

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Dual Regression for Efficient Hand Pose Estimation

Human Pose Estimation with Shape Aware Loss

Improving Robustness for Pose Estimation Via Stable Heatmap Regression

Bias-Compensated Integral Regression for Human Pose Estimation

Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks

Heatmap Distribution Matching for Human Pose Estimation

Robust 6-DoF Pose Estimation under Hybrid Constraints

3D Human Pose Estimation Base on Weighted Joint Loss.

Linear-Covariance Loss for End-to-End Learning of 6D Pose Estimation

Heterogeneous heatmap distillation framework based on unbiased alignment for lightweight human pose estimation

Multi-Person Pose Estimation with Accurate Heatmap Regression and Greedy Association

Hope: heatmap and offset for pose estimation

CbwLoss: Constrained Bidirectional Weighted Loss for Self-Supervised Learning of Depth and Pose

Low-resolution human pose estimation

Data Snooping for Pose Estimation Based on Generalized Errors-in-Variables Model

Discriminator Adaptive Wasserstein Hourglass Regression Network Depth Regularizer Loss 2 . 5 D s Loss rec s Loss rec t Loss 2 D t Loss wd Feature Similarity Weighting Skip Connection

Is 2D Heatmap Representation Even Necessary for Human Pose Estimation?

REDE: End-to-End Object 6D Pose Robust Estimation Using Differentiable Outliers Elimination

Hybrid model for Single-Stage Multi-Person Pose Estimation