Abstract:Exploiting the rich temporal information in human pose sequences to facilitate 3D pose estimation has garnered particular attention. While various learning architectures have been designed for temporal exploiting, these architectures are usually trained via the 3D pose loss independently imposed on every single frame, without explicit temporal signals introduced for supervision. This inevitably increases the difficulty of temporal exploiting, since the network must reason about the meaningful temporal information based on the non-temporal single-frame supervision first. Only then, the network can utilize this information to guide sequence modeling. Recently, some work introduce temporal smoothness as an explicit supervision signal, which makes the network more straightforwardly reaches the temporal information from the supervision signal, thus improving the temporal exploiting. However, the temporal smoothness only roughly measures the short-term temporal properties between adjacent frame pairs. In this work, we propose to generalize the supervision of temporal smoothness to temporal correlations, letting the network precisely consider more comprehensive temporal properties in sequences. We contribute two novel correlation-based loss functions, which adopt different strategies to respectively regularize the encoder and decoder sides of the network for temporal exploiting. Besides, we design a pre-training scheme to ensure a general convergence of existing pose estimators under our correlation losses. Experiments on three benchmarks demonstrate that our method can be compatible with different networks, improving their temporal exploiting ability to output more accurate and robust pose estimations.

Exploiting Temporal Correlations for 3D Human Pose Estimation

Temporal Constrained Feasible Subspace Learning for Human Pose Forecasting

Exploiting temporal information for 3D pose estimation

TCPFormer: Learning Temporal Correlation with Implicit Pose Proxy for 3D Human Pose Estimation

Temporal Feature Correlation for Human Pose Estimation in Videos

Exploring Temporal Consistency for Human Pose Estimation in Videos

An Improved 3D Human Pose Estimation Model Based on Temporal Convolution with Gaussian Error Linear Units

3D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Training

3D Human Pose Estimation with Spatio-Temporal Criss-Cross Attention

Learning Temporal-Spatial Contextual Adaptation for Three-Dimensional Human Pose Estimation

Spatial-temporal-spectral Transformer for 3D Human Pose Estimation.

3D Human Pose and Shape Reconstruction from Videos Via Confidence-Aware Temporal Feature Aggregation

Robust 3D Human Pose Estimation Model Based on Temporal Convolution

Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation

Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation

3D Human pose estimation from video via multi-scale multi-level spatial temporal features

Exploiting Spatial-Temporal Relationships for 3D Pose Estimation Via Graph Convolutional Networks.

3D Human Pose Estimation Via Non-Causal Retentive Networks

3D Human Pose Estimation Base on Weighted Joint Loss.

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

Spatio-temporal Tendency Reasoning for Human Body Pose and Shape Estimation from Videos