Abstract:Transformer-based methods have emerged as the golden standard in 2D-3D human pose estimation from video sequences, largely thanks to their powerful spatial–temporal feature encoders. In the past, researchers have made concerted efforts to engineer spatial and temporal encoders using transformer blocks. This approach involved a dramatic reshaping of the input, transforming it from mere joint information to dynamic joint trajectories. Despite this, the inherent limitations of the spatial–temporal structure have resulted in an inadequate acquisition and subsequent utilization of temporal information. In an attempt to rectify this prevalent issue, our paper proposes a new model, dubbed Spatial–Temporal-ReTemporal Transformer (i.e., STRFormer). This model ingeniously employs two separate temporal transformer blocks to extract the essential temporal motion information from video sequences. Intriguingly, one temporal transformer block is dedicated to the original video sequence, while the other concerns itself with the reversed order video. This novel approach allows for a more thorough investigation and utilization of temporal information from the video sequences. In order to alternate the processing of these two blocks effectively with the spatial block, we focus on maximizing the extraction of temporal domain information. This method leads to a more comprehensive understanding of the pose estimation and its evolution over time. Furthermore, we introduce a novel error metric, Mean Per-Joint Position Acceleration Error (i.e., MPJAE). This advanced metric takes into account the body part velocity in adjacent predicted frames, allowing for a more detailed evaluation of the predicted poses. We conduct extensive experiments on various open benchmarks to evaluate the effectiveness of our proposed model. The results demonstrate that our STRFormer, coupled with the MPJAE loss, achieves highly competitive results when compared with other state-of-the-art models. This illustrates its promising potential and practical applicability in 2D-3D human pose estimation tasks. We plan to release our code publicly for further research.

ICRFormer: an Improving Cos-Reweighting Transformer for 3D Human Pose Estimation in Video

EMHIFormer: an Enhanced Multi-Hypothesis Interaction Transformer for 3D Human Pose Estimation in Video

3D Human Pose Estimation with Spatial and Temporal Transformers

Motion Imitation of a Humanoid Robot Via Pose Estimation

CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation

EVOPOSE: A Recursive Transformer For 3D Human Pose Estimation With Kinematic Structure Priors

ConvFormer: Parameter Reduction in Transformer Models for 3D Human Pose Estimation by Leveraging Dynamic Multi-Headed Convolutional Attention

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

Cross-Space-Time 3D Human Body Pose Estimation Based on Transformer

HDFormer: High-order Directed Transformer for 3D Human Pose Estimation

Refined Temporal Pyramidal Compression-and-Amplification Transformer for 3D Human Pose Estimation

PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

Spatial-temporal-spectral Transformer for 3D Human Pose Estimation.

STRFormer: Spatial–Temporal–ReTemporal Transformer for 3D Human Pose Estimation

TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation

Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers

HSTFormer: Hierarchical Spatial-Temporal Transformers for 3D Human Pose Estimation

KTPFormer: Kinematics and Trajectory Prior Knowledge-Enhanced Transformer for 3D Human Pose Estimation

Frame-Padded Multiscale Transformer for Monocular 3D Human Pose Estimation

HEViTPose: High-Efficiency Vision Transformer for Human Pose Estimation