Abstract:The growing demand for mobile devices has generated interest in lightweight human pose estimation. Currently, lightweight estimation generally uses heatmap-based methods, which has demonstrated exceptional performance. However, their use of non-differentiable post-processing imposes considerable inference latencies. Conversely, integral-based approaches expedite the inference process by employing a soft-argmax operation but compromise in accuracy. Integrating explicit heatmap knowledge learned using the heatmap-based method into the implicit heatmap generated by the integral-based method, thereby combining the best of both worlds, offers a promising avenue. However, owing to the disparities in supervision and inference processes, the explicit and implicit heatmaps are heterogeneous. Consequently, direct transfer of knowledge presents difficulties in ensuring consistencies in heat value and location. In this paper, we propose a novel Heterogeneous Heatmap Distillation (HHD) framework that effectively tackles these challenges. The framework seamlessly integrates explicit heatmap knowledge that contains high-precision localization information into implicit heatmaps. The framework revolves around an unbiased heatmap alignment scheme encompassing two steps: heterogeneous heatmap normalization and unbiased cropping. Heterogeneous heatmap normalization separately normalizes the output feature maps of both the teacher and student models, alleviating potential heat value bias during the knowledge transfer. Unbiased cropping applies closed-form computation on the normalized teacher and student heatmap to eliminate location bias. Additionally, mirror expansion is implemented to handle potential cases wherein the cropped region extends beyond the image boundary. Extensive experiments demonstrate the efficiency and effectiveness of our methods on the MSCOCO and MPII datasets compared to other integral-based lightweight networks. Our source codes and pre-trained models are available at https://github.com/ducongju/HHD .

Unbiased Feature Position Alignment for Human Pose Estimation

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Adaptively Fusing Complete Multi-resolution Features for Human Pose Estimation.

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation.

Zero-Shot 3d Pose Estimation of Unseen Object by Two-Step Rgb-D Fusion

Mitigating imbalances in heterogeneous feature fusion for multi-class 6D pose estimation

The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation

A Multi-Channel Parallel Keypoint Fusion Framework for Human Pose Estimation

Reconstructing 3D human pose and shape from a single image and sparse IMUs

APP: Adaptive Pose Pooling for 3D Human Pose Estimation from Videos

FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation

3D Face Alignment Through Fusion of Head Pose Information and Features

Heterogeneous heatmap distillation framework based on unbiased alignment for lightweight human pose estimation

Feature Alignment in Anchor-Free Object Detection

BSAFusion: A Bidirectional Stepwise Feature Alignment Network for Unaligned Medical Image Fusion

Focus on Low-Resolution Information: Multi-Granular Information-Lossless Model for Low-Resolution Human Pose Estimation

MMDA: Multi-person marginal distribution awareness for monocular 3D pose estimation

Domain adaptive pose estimation via multi-level alignment

MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation

Learning a Robust Part-Aware Monocular 3D Human Pose Estimator via Neural Architecture Search