Abstract:3D human motion prediction; predicting future human poses in the basis of historically observed motion sequences, is a core task in computer vision. Thus far, it has been successfully applied to both autonomous driving and human–robot interaction. Previous research work has usually employed Recurrent Neural Networks (RNNs)-based models to predict future human poses. However, as previous works have amply demonstrated, RNN-based prediction models suffer from unrealistic and discontinuous problems in human motion prediction due to the accumulation of prediction errors. To address this, we propose a feed-forward, 3D skeleton-based model for human motion prediction. This model, the Spatial–Temporal Graph Convolutional Network (ST-GCN) model, automatically learns the spatial and temporal patterns of human motion from input sequences. This model overcomes the limitations of previous research approaches. Specifically, our ST-GCN model is based on an encoder-decoder architecture. The encoder consists of 5 ST-GCN modules, with each ST-GCN module consisting of a spatial GCN layer and a 2D convolution-based TCN layer, which facilitate the encoding of the spatio-temporal dynamics of human motion. Subsequently, the decoder, consisting of 5 TCN layers, exploits the encoded spatio-temporal representation of human motion to predict future human pose. We leveraged the ST-GCN model to perform extensive experiments on various large-scale human activity 3D pose datasets (Human3.6 M, AMASS, 3DPW) while adopting MPJPE (Mean Per Joint Position Error) as the evaluation metric. The experimental results demonstrate that our ST-GCN model outperforms the baseline models in both short-term (< 400 ms) and long-term (> 400 ms) predictions, thus yielding the best prediction results.

Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction

An Attractor-Guided Neural Networks for Skeleton-Based Human Motion Prediction

Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network.

3D Skeleton-Based Human Motion Prediction Using Spatial–temporal Graph Convolutional Network

Multitask Non-Autoregressive Model For Human Motion Prediction

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Parallel multi-stage rectification networks for 3D skeleton-based motion prediction

An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data

Directed Acyclic Graph Neural Network for Human Motion Prediction

Human Skeleton Tree Recurrent Neural Network with Joint Relative Motion Feature for Skeleton Based Action Recognition

Multi-level Motion Attention for Human Motion Prediction

Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction

Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networks

Multiscale Spatial and Temporal Learning for Human Motion Prediction

Spatio-Temporal Attention Deep Network for Skeleton Based View-Invariant Human Action Recognition

Skeleton-Based Human Action Recognition Using Spatial Temporal 3D Convolutional Neural Networks

Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic.

Human-Robot Collaboration Through a Multi-Scale Graph Convolution Neural Network With Temporal Attention

Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction

DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction

History Repeats Itself: Human Motion Prediction via Motion Attention