Abstract:Human motion capture from monocular videos has made significant progress in recent years. However, modern approaches often produce temporal artifacts, e.g. in form of jittery motion and struggle to achieve smooth and physically plausible motions. Explicitly integrating physics, in form of internal forces and exterior torques, helps alleviating these artifacts. Current state-of-the-art approaches make use of an automatic PD controller to predict torques and reaction forces in order to re-simulate the input kinematics, i.e. the joint angles of a predefined skeleton. However, due to imperfect physical models, these methods often require simplifying assumptions and extensive preprocessing of the input kinematics to achieve good performance. To this end, we propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting, inspired by a neural Kalman-filtering approach. We develop a control loop as a meta-PD controller to predict internal joint torques and external reaction forces, followed by a physics-based motion simulation. A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion, resulting in an optimal-state dynamics prediction. We show that this filtering step is crucial to provide an online supervision that helps balancing the shortcoming of the respective input motions, thus being important for not only capturing accurate global motion trajectories but also producing physically plausible human poses. The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics, compared to state of the art. The code is available on <a class="link-external link-https" href="https://github.com/cuongle1206/OSDCap" rel="external noopener nofollow">this https URL</a>

Neural Ordinary Differential Equation for Irregular Human Motion Prediction

Forecasting Distillation: Enhancing 3D Human Motion Prediction with Guidance Regularization

Robust Real-Time Motion Retargeting Via Neural Latent Prediction

Learning a Deep Motion Interpolation Network for Human Skeleton Animations

Velocity-to-velocity human motion forecasting

Multitask Non-Autoregressive Model For Human Motion Prediction

Existence Is Chaos: Enhancing 3D Human Motion Prediction with Uncertainty Consideration

DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction

Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction

A Neural Temporal Model for Human Motion Prediction

A Continuous Learning Approach for Probabilistic Human Motion Prediction

A Tensor Based Regression Approach for Human Motion Prediction

An Attractor-Guided Neural Networks for Skeleton-Based Human Motion Prediction

A Data-Efficient Approach for Long-Term Human Motion Prediction Using Maps of Dynamics

3D Human motion anticipation and classification

Motion Forecasting Network (mofcnet): IMU-Based Human Motion Forecasting for Hip Assistive Exoskeleton

A Stochastic Conditioning Scheme for Diverse Human Motion Prediction

AnyPose: Anytime 3D Human Pose Forecasting via Neural Ordinary Differential Equations

Human Motion Prediction under Unexpected Perturbation

Human Motion Prediction Using Manifold-Aware Wasserstein GAN

Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos