Abstract:Extracting physically plausible 3D human motion from videos is a critical task. Although existing simulation-based motion imitation methods can enhance the physical quality of daily motions estimated from monocular video capture, extending this capability to high-difficulty motions remains an open challenge. This can be attributed to some flawed motion clips in video-based motion capture results and the inherent complexity in modeling high-difficulty motions. Therefore, sensing the advantage of segmentation in localizing human body, we introduce a mask-based motion correction module (MCM) that leverages motion context and video mask to repair flawed motions, producing imitation-friendly motions; and propose a physics-based motion transfer module (PTM), which employs a pretrain and adapt approach for motion imitation, improving physical plausibility with the ability to handle in-the-wild and challenging motions. Our approach is designed as a plug-and-play module to physically refine the video motion capture results, including high-difficulty in-the-wild motions. Finally, to validate our approach, we collected a challenging in-the-wild test set to establish a benchmark, and our method has demonstrated effectiveness on both the new benchmark and existing public <a class="link-external link-http" href="http://datasets.https" rel="external noopener nofollow">this http URL</a>://physicalmotionrestoration.<a class="link-external link-http" href="http://github.io" rel="external noopener nofollow">this http URL</a>

A physically-based algorithm for motion mapping

A New Motion Editing Algorithm Based On Motion Capture Data

A Motion Retargeting Algorithm Based on Model Simplification

Motion retargeting based on physical constraint

Space-Time Constraints Based Motion Editing and Motion Retargeting

A 3D Human Motion Refinement Method Based on Sparse Motion Bases Selection.

Editing Human Motion Path

A New Algorithm for Human Motion Capture via 3D Active Contours

Data-Driven Motion Exaggeration

Motion Processing in Tight-Clothing Based Motion Capture

Markerless Human Body Motion Capture Using Multiple Cameras

Human Body Motion Capture Via 3d Graph-Cuts

Physical Based Motion Reconstruction from Videos Using Musculoskeletal Model

Markerless Human Body Motion Capture Using Markov Random Field and Dynamic Graph Cuts

Performance-based Control Interfaces Using Mixture of Factor Analyzers

Trajectory-Mapping Method Based On Feature Extraction For Motion Retargeting And It'S Application

A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motions

Approach of Scattered Data Processing for Optical Motion Capture

Editing and Synthesis of 3D Facial Animation by Motion Propagation and Isomap Analysis

Model-Based Markerless Human Body Motion Capture using Multiple Cameras

New multi-view human motion capture framework