Abstract:Recent advances in learning reusable motion priors have demonstrated their effectiveness in generating naturalistic behaviors. In this paper, we propose a new learning framework in this paradigm for controlling physics-based characters with significantly improved motion quality and diversity over existing state-of-the-art methods. The proposed method uses reinforcement learning (RL) to initially track and imitate life-like movements from unstructured motion clips using the discrete information bottleneck, as adopted in the Vector Quantized Variational AutoEncoder (VQ-VAE). This structure compresses the most relevant information from the motion clips into a compact yet informative latent space, i.e., a discrete space over vector quantized codes. By sampling codes in the space from a trained categorical prior distribution, high-quality life-like behaviors can be generated, similar to the usage of VQ-VAE in computer vision. Although this prior distribution can be trained with the supervision of the encoder's output, it follows the original motion clip distribution in the dataset and could lead to imbalanced behaviors in our setting. To address the issue, we further propose a technique named prior shifting to adjust the prior distribution using curiosity-driven RL. The outcome distribution is demonstrated to offer sufficient behavioral diversity and significantly facilitates upper-level policy learning for downstream tasks. We conduct comprehensive experiments using humanoid characters on two challenging downstream tasks, sword-shield striking and two-player boxing game. Our results demonstrate that the proposed framework is capable of controlling the character to perform considerably high-quality movements in terms of behavioral strategies, diversity, and realism. Videos, codes, and data are available at <a class="link-external link-https" href="https://tencent-roboticsx.github.io/NCP/" rel="external noopener nofollow">this https URL</a>.

Character Controllers Using Motion VAEs

ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters

MuscleVAE: Model-Based Controllers of Muscle-Actuated Characters

Task-Generic Hierarchical Human Motion Prior using VAEs

Responsive Action Generation By Physically-Based Motion Retrieval And Adaptation

VMP: Versatile Motion Priors for Robustly Tracking Motion on Physical Characters

A Motion-based User Interface for the Control of Virtual Humans Performing Sports.

A language‐directed virtual human motion generation approach based on musculoskeletal models

Optimization algorithm of an artificial neural network-based controller and simulation method for animated virtual idol characters

SFV: Reinforcement Learning of Physical Skills from Videos

Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

MoVEInt: Mixture of Variational Experts for Learning Human-Robot Interactions from Demonstrations

Example-driven Virtual Cinematography by Learning Camera Behaviors

Neural Categorical Priors for Physics-Based Character Control

Automatic Generation of Human Animation Based on Motion Programming: Virtual Humans and Social Agents

Action2Motion: Conditioned Generation of 3D Human Motions

Data-Driven Reinforcement Learning for Virtual Character Animation Control

An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural Networks

Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control