Abstract:We propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion. It takes reference motion data, caption and text prompts as inputs, realizing the natural language motion controller through three components: constructing an action‐caption data pool, learning the control policies for imitating the motion, and semantic matching selection. The development of the systems capable of synthesizing natural and life‐like motions for virtual characters has long been a central focus in computer animation. It needs to generate high‐quality motions for characters and provide users with a convenient and flexible interface for guiding character motions. In this work, we propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion, which lays the foundation for the development of language‐directed controllers in physics‐based character animation. First, we construct a simplified model of musculoskeletal dynamics for the virtual character. Subsequently, we propose a hierarchical control framework consisting of a trajectory tracking layer and a muscle control layer, obtaining the optimal control policy for imitating the reference motions through the training. We design a multi‐policy aggregation controller based on large language models, which selects the motion policy with the highest similarity to user text commands from the action‐caption data pool, facilitating natural language‐based control of virtual character motions. Experimental results demonstrate that the proposed approach not only generates high‐quality motions highly resembling reference motions but also enables users to effectively guide virtual characters to perform various motions via natural language instructions.

Inverse kinematics based on statistical learning

Pose Synthesis of Virtual Character Based on Statistical Learning

Gaussian Process Latent Variable Models for Inverse Kinematics

Motion Editing for Quadruped Characters Via Latent Frequency Embedding

A Joint Physical Constraints Avoidance Method for Inverse Kinematics Problem of Redundant Humanoid Manipulator

Inverse kinematics using dynamic joint parameters: inverse kinematics animation synthesis learnt from sub-divided motion micro-segments

Inverse Kinematics Solution for Robot Manipulator Based on Neural Network under Joint Subspace.

A constrained inverse kinematics technique for real-time motion capture animation

Nonlinear Dimensional Reduction and Generation of Human Motion

Group Animations Based on Machine Learning

A Real-Time Inverse Kinematics Algorithm for Human Motion Modeling

Human Motion Capture Using Wireless Inertial Sensors

A Method for Character Animation Generation and Control

Statistical learning based facial animation

Dimensionality Reduction And Generation Of Human Motion

Learning Human Motion Prediction Via Stochastic Differential Equations

Learning Graphical Model For Human Motion Characterization Using Genetic Optimization

Learning-based pose edition for efficient and interactive design

Space-time constrained optimization for deep locomotion controller

Learning Inverse Statics Models Efficiently

A language‐directed virtual human motion generation approach based on musculoskeletal models