Abstract:We propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion. It takes reference motion data, caption and text prompts as inputs, realizing the natural language motion controller through three components: constructing an action‐caption data pool, learning the control policies for imitating the motion, and semantic matching selection. The development of the systems capable of synthesizing natural and life‐like motions for virtual characters has long been a central focus in computer animation. It needs to generate high‐quality motions for characters and provide users with a convenient and flexible interface for guiding character motions. In this work, we propose a language‐directed virtual human motion generation approach based on musculoskeletal models to achieve interactive and higher‐fidelity virtual human motion, which lays the foundation for the development of language‐directed controllers in physics‐based character animation. First, we construct a simplified model of musculoskeletal dynamics for the virtual character. Subsequently, we propose a hierarchical control framework consisting of a trajectory tracking layer and a muscle control layer, obtaining the optimal control policy for imitating the reference motions through the training. We design a multi‐policy aggregation controller based on large language models, which selects the motion policy with the highest similarity to user text commands from the action‐caption data pool, facilitating natural language‐based control of virtual character motions. Experimental results demonstrate that the proposed approach not only generates high‐quality motions highly resembling reference motions but also enables users to effectively guide virtual characters to perform various motions via natural language instructions.

The Role of Control Based on Three-Dimensional Human Model

Three-Dimensional Human Modeling and Dynamics Simulation Based on ADAMS

Animating Scanned Human Models.

A Realization Method of 3D Virtual Human Animation Driven By Video

A Design for 3D Animation Based on Real-Time Data Collections

Design and Realization Method of Robot Dance Motion Based On Virtual Human

Physically Based Real-Time Human Figure Animation

Modeling human-like autonomous behaviors and movements of virtual humans in real-time virtual environment

Research of Skeletal-Based Virtual Human Animation Using Cal3D

A Survey on Human Performance Capture and Animation

Tight-Clothing Based Human Animation Research

Real-time hierarchical movement animation control for autonomous virtual humans

Human Modeling of Virtual Environment Based on Multigen Technology

Editable Method to 3D Human Skinned Mesh Animation Synthesis

Method of Driving a 3D Model Based on Vision-Based Motion Capture Data

A language‐directed virtual human motion generation approach based on musculoskeletal models

State of the Art of the Research on Human-body Modeling and Skin Deformation in Computer Animation

3D Animation Production and Generation Based on Human Body Base 3D Models

A Video-driven Approach to 3D Human Animation Synthesis

Biomechanics Based 3D Human Motion Model

Reconstructing Three-Dimensional Human Poses: A Combined Approach of Iterative Calculation on Skeleton Model and Conformal Geometric Algebra.