Abstract:Recent advances in learning reusable motion priors have demonstrated their effectiveness in generating naturalistic behaviors. In this paper, we propose a new learning framework in this paradigm for controlling physics-based characters with significantly improved motion quality and diversity over existing state-of-the-art methods. The proposed method uses reinforcement learning (RL) to initially track and imitate life-like movements from unstructured motion clips using the discrete information bottleneck, as adopted in the Vector Quantized Variational AutoEncoder (VQ-VAE). This structure compresses the most relevant information from the motion clips into a compact yet informative latent space, i.e., a discrete space over vector quantized codes. By sampling codes in the space from a trained categorical prior distribution, high-quality life-like behaviors can be generated, similar to the usage of VQ-VAE in computer vision. Although this prior distribution can be trained with the supervision of the encoder's output, it follows the original motion clip distribution in the dataset and could lead to imbalanced behaviors in our setting. To address the issue, we further propose a technique named prior shifting to adjust the prior distribution using curiosity-driven RL. The outcome distribution is demonstrated to offer sufficient behavioral diversity and significantly facilitates upper-level policy learning for downstream tasks. We conduct comprehensive experiments using humanoid characters on two challenging downstream tasks, sword-shield striking and two-player boxing game. Our results demonstrate that the proposed framework is capable of controlling the character to perform considerably high-quality movements in terms of behavioral strategies, diversity, and realism. Videos, codes, and data are available at <a class="link-external link-https" href="https://tencent-roboticsx.github.io/NCP/" rel="external noopener nofollow">this https URL</a>.

CALM: Conditional Adversarial Latent Models for Directable Virtual Characters

Taming Diffusion Probabilistic Models for Character Control

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

Responsive Action Generation By Physically-Based Motion Retrieval And Adaptation

A Motion-based User Interface for the Control of Virtual Humans Performing Sports.

CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion

DReCon

SFV: Reinforcement Learning of Physical Skills from Videos

Real-Time Stylized Humanoid Behavior Control through Interaction and Synchronization

Gated Neural Network Framework for Interactive Character Control

Learning Uniformly Distributed Embedding Clusters of Stylistic Skills for Physically Simulated Characters

C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters

Character Controllers Using Motion VAEs

MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting

Perpetual Humanoid Control for Real-time Simulated Avatars

Interactive Character Control with Auto-Regressive Motion Diffusion Models

Multi-agent reinforcement learning for character control

Neural Categorical Priors for Physics-Based Character Control

Towards smarter animated embodied agents

Simulation and Retargeting of Complex Multi-Character Interactions