Abstract:Analyses of human motion kinematics have achieved tremendous advances. However, the production mechanism, known as human dynamics, is still undercovered. In this paper, we aim to push data-driven human dynamics understanding forward. We identify a major obstacle to this as the heterogeneity of existing human motion understanding efforts. Specifically, heterogeneity exists in not only the diverse kinematics representations and hierarchical dynamics representations but also in the data from different domains, namely biomechanics and reinforcement learning. With an in-depth analysis of the existing heterogeneity, we propose to emphasize the beneath homogeneity: all of them represent the homogeneous fact of human motion, though from different perspectives. Given this, we propose Homogeneous Dynamics Space (HDyS) as a fundamental space for human dynamics by aggregating heterogeneous data and training a homogeneous latent space with inspiration from the inverse-forward dynamics procedure. Leveraging the heterogeneous representations and datasets, HDyS achieves decent mapping between human kinematics and dynamics. We demonstrate the feasibility of HDyS with extensive experiments and applications. The project page is <a class="link-external link-https" href="https://foruck.github.io/HDyS" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is **how to deeply understand human dynamics through data - driven methods**. Specifically, the author aims to establish a two - way mapping relationship between human kinematics and dynamics in order to better understand the generation mechanism of human motion. However, there is a major obstacle in existing research: there is significant heterogeneity among human motion data from different sources and representations, which hinders the unified understanding of human dynamics. ### Specific manifestations of the problem 1. **Representation Heterogeneity**: - There are various kinematic representations, including markers, skeletal keypoints, joint angles, and SMPL model parameters, etc. - Dynamics representations also vary, such as joint torques, muscle actions, and surface electromyography (sEMG) signals, etc. 2. **Domain Heterogeneity**: - Biomechanics data usually comes from laboratory environments, with precise computational models and real - motion data, but is limited to simple motion types. - Data generated by reinforcement learning (RL) is more diverse, but there may be deviations in physical simulations, resulting in the sim2real domain gap. ### Solutions To solve these problems, the author proposes the **Homogeneous Dynamics Space (HDyS)** through the following methods: - **Aggregating heterogeneous data**: Collect data in multiple kinematic and dynamic representations from different data sources (such as biomechanics data and reinforcement learning data). - **Constructing a unified latent space**: Design a multi - auto - encoder structure to map heterogeneous kinematic and dynamic representations into a common latent space. - **Supervised training**: Use reconstruction loss and alignment loss to supervise the learning process of HDyS, ensuring that knowledge among different representations can be effectively integrated. ### Experimental verification The author demonstrates the effectiveness of HDyS through extensive experiments, including the evaluation of human inverse dynamics and its performance in downstream applications, such as ground reaction force prediction, biomechanical human simulation, and physical simulation character control. ### Summary The main contributions of this paper are: 1. Analyzing the heterogeneity problem that hinders in - depth understanding of human dynamics. 2. Proposing a unified and reusable space HDyS, and mining the homogeneous knowledge behind it by integrating heterogeneous data. 3. Verifying the feasibility of extracting homogeneity from heterogeneity through a large number of experiments and demonstrating the potential of HDyS in multiple application scenarios. Through these efforts, the author hopes to promote the understanding of human dynamics and provide new tools and methods for future research.

Homogeneous Dynamics Space for Heterogeneous Humans

ImDy: Human Inverse Dynamics from Imitated Observations

Revisit Human-Scene Interaction via Space Occupancy

Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space

HSPACE: Synthetic Parametric Humans Animated in Complex Environments

Dynamic Future Net: Diversified Human Motion Generation

DiverseMotion: Towards Diverse Human Motion Generation Via Discrete Diffusion

Synthesizing Diverse Human Motions in 3D Indoor Scenes

DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics

D&D: Learning Human Dynamics from Dynamic Camera

TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generation

A Data-Efficient Approach for Long-Term Human Motion Prediction Using Maps of Dynamics

Automatic Human Scene Interaction through Contact Estimation and Motion Adaptation

Dynamic Motion Transition: A Hybrid Data-driven and Model-driven Method for Human Pose Transitions

Physics-based Human Motion Estimation and Synthesis from Videos

How Do We Move: Modeling Human Movement with System Dynamics

Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions

HUMOS: Human Motion Model Conditioned on Body Shape

Optimal-State Dynamics Estimation for Physics-based Human Motion Capture from Videos

HiSC4D: Human-centered interaction and 4D Scene Capture in Large-scale Space Using Wearable IMUs and LiDAR