Abstract:Human motion prediction is an increasingly interesting topic in computer vision and robotics. In this paper, we propose a new end-to-end feedforward network, TrajectoryCNN, to predict future poses. Compared with the most existing methods, we introduce a new trajectory space and focus on modeling motion dynamics of the input sequence with coupled spatio-temporal features, dynamic local-global features, and global temporal co-occurrence features in the new space. Specifically, the coupled spatio-temporal features describe the spatial and temporal structural information hidden in a natural human motion sequence, which can be easily mined using CNN by simultaneously covering the spatial and temporal dimensions of the sequence with the convolutional filters. The dynamic local-global features encode different correlations among joint trajectories of human motion (i.e. strong correlations among joint trajectories of one part and weak correlations among joint trajectories of different parts), which can be captured by stacking multiple residual trajectory blocks and incorporating our skeletal representation. The global temporal co-occurrence features represent different importance of different input poses to mine the motion dynamics for predicting future poses, which can be obtained automatically by learning free parameters for each pose with our TrajectoryCNN. Finally, we predict future poses with the captured motion dynamic features in a non-recursive manner. Extensive experiments show that our method achieves state-of-the-art performance on five benchmarks (e.g. Human3.6M, CMU-Mocap, 3DPW, G3D, and FNTU), which demonstrates the effectiveness of our proposed method. The code is available at https://github.com/lily2lab/TrajectoryCNN.git.

Pose-Transformed Equivariant Network for 3D Point Trajectory Prediction

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Trajectorylet-Net: a novel framework for pose prediction based on trajectorylet descriptors.

An Adaptive Viewpoint Transformation Network for 3D Human Pose Estimation

Equivariant Single View Pose Prediction Via Induced and Restricted Representations

Equivariant Point Network for 3D Point Cloud Analysis

EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning

Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction

TrajectoryCNN: A New Spatio-Temporal Feature Learning Network for Human Motion Prediction

Achieving view-distance and -angle invariance in motion prediction using a simple network

Equivariant Point Cloud Analysis via Learning Orientations for Message Passing

3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction

Trajectory Prediction using Equivariant Continuous Convolution

Motion Guided 3D Pose Estimation from Videos

Motion Prediction Using Trajectory Cues

Towards Accurate 3D Human Motion Prediction from Incomplete Observations

KVNet: an Iterative 3D Keypoints Voting Network for Real-Time 6-Dof Object Pose Estimation

RECO: Rotation Equivariant COnvolutional Neural Network for Human Trajectory Forecasting.

Keypoint-Based Disentangled Pose Network for Category-Level 6-D Object Pose Tracking.

SE(3) Equivariant Graph Neural Networks with Complete Local Frames

Multiple-level Point Embedding for Solving Human Trajectory Imputation with Prediction