Abstract:The transformer architecture has consistently achieved cutting-edge performance in the task of 2D to 3D lifting human pose estimation. Despite advances in transformer-based methods they still suffer from issues related to sequential data processing, addressing depth ambiguity, and effective handling of sensitive noisy data. As a result, transformer encoders encounter difficulties in precisely estimating human positions. To solve this problem, a novel multi-transformer encoder with a multiple-hypothesis aggregation (MHAFormer) module is proposed in this study. To do this, a diffusion module is first introduced that generates multiple 3D pose hypotheses and gradually distributes Gaussian noise to ground truth 3D poses. Subsequently, the denoiser is employed within the diffusion module to restore the feasible 3D poses by leveraging the information from the 2D keypoints. Moreover, we propose the multiple-hypothesis aggregation with a join-level reprojection (MHAJR) approach that redesigns the 3D hypotheses into the 2D position and selects the optimal hypothesis by considering reprojection errors. In particular, the multiple-hypothesis aggregation approach tackles depth ambiguity and sequential data processing by considering various possible poses and combining their strengths for a more accurate final estimation. Next, we present the improved spatial-temporal transformers encoder that can help to improve the accuracy and reduce the ambiguity of 3D pose estimation by explicitly modeling the spatial and temporal relationships between different body joints. Specifically, the temporal-transformer encoder introduces the temporal constriction & proliferation (TCP) attention mechanism and the feature aggregation refinement module (FAR) into the refined temporal constriction & proliferation (RTCP) transformer, which enhances intra-block temporal modeling and further refines inter-block feature interaction. Finally, the superiority of the proposed approach is demonstrated through comparison with existing methods using the Human3.6M and MPI-INF-3DHP benchmark datasets.

SPGformer: Serial-Parallel Hybrid GCN-Transformer with Graph-Oriented Encoder for 2D-to-3d Human Pose Estimation

SPGformer: Serial–Parallel Hybrid GCN-Transformer With Graph-Oriented Encoder for 2-D-to-3-D Human Pose Estimation

Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution.

DGFormer: Dynamic Graph Transformer for 3D Human Pose Estimation

Graph and Skipped Transformer: Exploiting Spatial and Temporal Modeling Capacities for Efficient 3D Human Pose Estimation

Global and Local Spatio-Temporal Encoder for 3D Human Pose Estimation

GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video

A Simple yet Effective 2D-3D Lifting Method for Monocular 3D Human Pose Estimation.

STGFormer: Spatio-Temporal GraphFormer for 3D Human Pose Estimation in Video

3D Human Pose Estimation Via Graph Extended Spatio-Temporal Convolutional Network

MSMB-GCN: Multi-scale Multi-branch Fusion Graph Convolutional Networks for 3D Human Pose Estimation

PoseGTAC: Graph Transformer Encoder-Decoder with Atrous Convolution for 3D Human Pose Estimation

HPGCN: Hierarchical Poselet-Guided Graph Convolutional Network for 3D Pose Estimation

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

Simplified-attention Enhanced Graph Convolutional Network for 3D human pose estimation

Exploiting multi-transformer encoder with multiple-hypothesis aggregation via diffusion model for 3D human pose estimation

A Geometric Knowledge Oriented Single-Frame 2D-to-3D Human Absolute Pose Estimation Method

Parallel‐branch Network for 3D Human Pose and Shape Estimation in Video

Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Training

MLP-JCG: Multi-Layer Perceptron with Joint-Coordinate Gating for Efficient 3D Human Pose Estimation

A multi-granular joint tracing transformer for video-based 3D human pose estimation