Abstract:We propose a Context-aware Feature Transformer Network (CaFTNet), a novel network for human pose estimation. To address the issue of limited modeling of global dependencies in convolutional neural networks, we design the Transformerneck to strengthen the expressive power of features. Transformerneck directly substitutes 3×3 convolution in the bottleneck of HRNet with a Contextual Transformer (CoT) block while reducing the complexity of the network. Specifically, the CoT first produces keys with static contextual information through 3×3 convolution. Then, relying on query and contextualization keys, dynamic contexts are generated through two concatenated 1×1 convolutions. Static and dynamic contexts are eventually fused as an output. Additionally, for multi-scale networks, in order to further refine the features of the fusion output, we propose an Attention Feature Aggregation Module (AFAM). Technically, given an intermediate input, the AFAM successively deduces attention maps along the channel and spatial dimensions. Then, an adaptive refinement module (ARM) is exploited to activate the obtained attention maps. Finally, the input undergoes adaptive feature refinement through multiplication with the activated attention maps. Through the above procedures, our lightweight network provides powerful clues for the detection of keypoints. Experiments are performed on the COCO and MPII datasets. The model achieves a 76.2 AP on the COCO val2017 dataset. Compared to other methods with a CNN as the backbone, CaFTNet has a 72.9% reduced number of parameters. On the MPII dataset, our method uses only 60.7% of the number of parameters, acquiring similar results to other methods with a CNN as the backbone.

Adaptively Fusing Complete Multi-resolution Features for Human Pose Estimation.

Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Full-Resolution Encoder-Decoder Networks with Multi-Scale Feature Fusion for Human Pose Estimation

Complementary Feature Pyramid Network for Human Pose Estimation

Lightweight high-resolution network based on adaptive cross-dimensional weighting for human pose estimation

Learning high resolution reservation for human pose estimation

Multi-Stage HRNet: Multiple Stage High-Resolution Network for Human Pose Estimation

Deep High-Resolution Representation Learning For Human Pose Estimation

Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention

An improved lightweight high-resolution network based on multi-dimensional weighting for human pose estimation

Focus on Low-Resolution Information: Multi-Granular Information-Lossless Model for Low-Resolution Human Pose Estimation

MSRT: multi-scale representation transformer for regression-based human pose estimation

High-Resolution Representations for Labeling Pixels and Regions.

A Lightweight Context-Aware Feature Transformer Network for Human Pose Estimation

Human Pose Estimation Based on Efficient and Lightweight High-Resolution Network (EL-HRNet)

Multi-Scale Structure-Aware Network for Human Pose Estimation

A-HRNet: Attention Based High Resolution Network for Human Pose Estimation

Combining detailed appearance and multi-scale representation: a structure-context complementary network for human pose estimation