Abstract:We propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion. It takes reference motion data, caption and text prompts as inputs, realizing the natural language motion controller through three components: constructing an action‐caption data pool, learning the control policies for imitating the motion, and semantic matching selection. The development of the systems capable of synthesizing natural and life‐like motions for virtual characters has long been a central focus in computer animation. It needs to generate high‐quality motions for characters and provide users with a convenient and flexible interface for guiding character motions. In this work, we propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion, which lays the foundation for the development of language‐directed controllers in physics‐based character animation. First, we construct a simplified model of musculoskeletal dynamics for the virtual character. Subsequently, we propose a hierarchical control framework consisting of a trajectory tracking layer and a muscle control layer, obtaining the optimal control policy for imitating the reference motions through the training. We design a multi‐policy aggregation controller based on large language models, which selects the motion policy with the highest similarity to user text commands from the action‐caption data pool, facilitating natural language‐based control of virtual character motions. Experimental results demonstrate that the proposed approach not only generates high‐quality motions highly resembling reference motions but also enables users to effectively guide virtual characters to perform various motions via natural language instructions.

Motion Control and Walking Simulation of Virtual Human Based on Biomechanics

Three-Dimensional Human Modeling and Dynamics Simulation Based on ADAMS

Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning

Dynamic Virtual Human Control Guided by Kinetic Energy Model

Walking Simulation of Biped Robot Based on Virtual Prototype

Research on Motion Simulation of Biped Walking Robot Based on Virtual Prototype Technology

Modeling and Control of Passive Dynamic Walking Robot with Humanoid Gait

Human Walking Mechanism and Gait Planning of Humanoid Robots

Motion control simulation based on VR for humanoid robot

Research and Implement of Virtual Human's Walking Model in Maintenance Simulation

Gait Generation and Control of Biped Robot with Moving Torso Based on Virtual Constraint

A Guide for Human Walking Model and Control — Insights from Mechanical Property Analysis of Human Walking

Research of Walking and Running Locomotion Control Methods for Virtual Humans

Muscle‐driven Virtual Human Motion Generation Approach Based on Deep Reinforcement Learning

The Simulation Model of Humanoid Robot Biped Locomotion Base on Simulink

Dynamic Walking Control of Underactuated 3D Biped Robot

Virtual Model Control for Quadruped Robots.

Human-Like Walking with Heel off and Toe Support for Biped Robot

A language‐directed virtual human motion generation approach based on musculoskeletal models

Walking Mechanism And Kinematic Analysis Of Humanoid Robot

Humanoid Robot 3- D Motion Simulation for Hardware Realization