Abstract:Human motion capture from monocular videos has made significant progress in recent years. However, modern approaches often produce temporal artifacts, e.g. in form of jittery motion and struggle to achieve smooth and physically plausible motions. Explicitly integrating physics, in form of internal forces and exterior torques, helps alleviating these artifacts. Current state-of-the-art approaches make use of an automatic PD controller to predict torques and reaction forces in order to re-simulate the input kinematics, i.e. the joint angles of a predefined skeleton. However, due to imperfect physical models, these methods often require simplifying assumptions and extensive preprocessing of the input kinematics to achieve good performance. To this end, we propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting, inspired by a neural Kalman-filtering approach. We develop a control loop as a meta-PD controller to predict internal joint torques and external reaction forces, followed by a physics-based motion simulation. A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion, resulting in an optimal-state dynamics prediction. We show that this filtering step is crucial to provide an online supervision that helps balancing the shortcoming of the respective input motions, thus being important for not only capturing accurate global motion trajectories but also producing physically plausible human poses. The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics, compared to state of the art. The code is available on <a class="link-external link-https" href="https://github.com/cuongle1206/OSDCap" rel="external noopener nofollow">this https URL</a>

Sampling-based Contact-Rich Motion Control.

Improving Sampling‐based Motion Control

Unified Motion/Force/Impedance Control for Manipulators in Unknown Contact Environments Based on Robust Model-Reaching Approach

Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture

On‐line Real‐time Physics‐based Predictive Motion Control with Balance Recovery

Efficient Sampling of Transition Constraints for Motion Planning under Sliding Contacts

Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

Kinematic Motion Retargeting for Contact-Rich Anthropomorphic Manipulations

Optimal-State Dynamics Estimation for Physics-based Human Motion Capture from Videos

Real-time Physics-based Motion Capture with Sparse Sensors

Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture

Reactive collision-free motion generation in joint space via dynamical systems and sampling-based MPC

Perpetual Humanoid Control for Real-time Simulated Avatars

Sampling-Based Model Predictive Control for Dexterous Manipulation on a Biomimetic Tendon-Driven Hand

STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation

A Convex Model of Momentum Dynamics for Multi-Contact Motion Generation

Contact-Implicit Model Predictive Control: Controlling Diverse Quadruped Motions Without Pre-Planned Contact Modes or Trajectories

A Human Motion Capture System Based on Inertial Sensing and a Complementary Filter

PartwiseMPC: Interactive Control of Contact‐Guided Motions