Abstract:Capturing cross-pose correlation from a sequence of frame-level 2D poses is essential for 3D human pose estimation (3D-HPE) in the video. Recent studies have shown the promising potential of modeling the pose relation with feature-mixing operations on the temporal domain. However, they seldom consider the interaction across poses in the frequency domain. This paper studies a Frequency-Temporal Collaborative Module (FTCM) to explore the feasibility of encoding the cross-pose correlations in both frequency and temporal domains. FTCM aims to jointly capture the global and local cross-pose correlations with a more lightweight network model. Specifically, FTCM splits the pose features into two groups along the channel dimension and separately models the frequency and temporal interactions across poses with different feature-mixing operations in parallel. To achieve this goal, we purposely design two pose-mixing units, i.e., the frequency pose-mixing (FPM) and the temporal pose-mixing (TPM). Particularly, FPM is designed to reap the global correlations among different pose frequencies with the representation obtained by converting the original pose signals with Fast Fourier transform (FFT). Unlike the pose-mixing used by previous methods like Transformers that influences an individual pose with all other poses, TPM locally calibrates the pose with dynamics aggregated within several adjacent poses in the temporal domain, explicitly weighting neighboring poses more with respect to the far-away ones so as to enforce a strict locality constraint. Besides, the group strategy significantly reduces the model complexity. To verify the effectiveness of FTCM, we conduct extensive experiments on two benchmarks (i.e., Human3.6M and MPI-INF-3DHP). Experimental results not only exhibit favorable accuracy/complexity trade-offs of our FTCM but also show superior or comparable performance to state-of-the-art methods on both datasets. The code and model are publicly available at: https://github.com/zhenhuat/FTCM.

FTCM: Frequency-Temporal Collaborative Module for Efficient 3D Human Pose Estimation in Video

FreqHPT: Frequency-aware Attention and Flow Fusion for Human Pose Transfer.

3D Human Pose Estimation with Spatio-Temporal Criss-Cross Attention

PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation

PA-Pose: Partial Point Cloud Fusion Based on Reliable Alignment for 6D Pose Tracking

FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions

Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Video

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

Double-chain Constraints for 3D Human Pose Estimation in Images and Videos

Temporal Feature Correlation for Human Pose Estimation in Videos

Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation.

Efficient Hierarchical Multi-view Fusion Transformer for 3D Human Pose Estimation

3D Human pose estimation from video via multi-scale multi-level spatial temporal features

MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video

Center point to pose: Multiple views 3D human pose estimation for multi-person

TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation

3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

MOTPose: Multi-object 6D Pose Estimation for Dynamic Video Sequences using Attention-based Temporal Fusion

Towards Precise 3D Human Pose Estimation with Multi-Perspective Spatial-Temporal Relational Transformers

APP: Adaptive Pose Pooling for 3D Human Pose Estimation from Videos

Unified End-to-End YOLOv5-HR-TCM Framework for Automatic 2D/3D Human Pose Estimation for Real-Time Applications