Abstract:Recent advances in learning reusable motion priors have demonstrated their effectiveness in generating naturalistic behaviors. In this paper, we propose a new learning framework in this paradigm for controlling physics-based characters with significantly improved motion quality and diversity over existing state-of-the-art methods. The proposed method uses reinforcement learning (RL) to initially track and imitate life-like movements from unstructured motion clips using the discrete information bottleneck, as adopted in the Vector Quantized Variational AutoEncoder (VQ-VAE). This structure compresses the most relevant information from the motion clips into a compact yet informative latent space, i.e., a discrete space over vector quantized codes. By sampling codes in the space from a trained categorical prior distribution, high-quality life-like behaviors can be generated, similar to the usage of VQ-VAE in computer vision. Although this prior distribution can be trained with the supervision of the encoder's output, it follows the original motion clip distribution in the dataset and could lead to imbalanced behaviors in our setting. To address the issue, we further propose a technique named prior shifting to adjust the prior distribution using curiosity-driven RL. The outcome distribution is demonstrated to offer sufficient behavioral diversity and significantly facilitates upper-level policy learning for downstream tasks. We conduct comprehensive experiments using humanoid characters on two challenging downstream tasks, sword-shield striking and two-player boxing game. Our results demonstrate that the proposed framework is capable of controlling the character to perform considerably high-quality movements in terms of behavioral strategies, diversity, and realism. Videos, codes, and data are available at <a class="link-external link-https" href="https://tencent-roboticsx.github.io/NCP/" rel="external noopener nofollow">this https URL</a>.

Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning

Learning to schedule control fragments for physics-based character simulation and robots using deep q-learning

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Guided Learning of Control Graphs for Physics-Based Characters

Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning

Hierarchical Planning and Control for Box Loco-Manipulation

Complex Locomotion Skill Learning via Differentiable Physics

Neural Categorical Priors for Physics-Based Character Control

Strategy and Skill Learning for Physics-based Table Tennis Animation

Learning Uniformly Distributed Embedding Clusters of Stylistic Skills for Physically Simulated Characters

DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills

Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics

Fragment-Based Responsive Character Motion for Interactive Games

Learning Reduced-Order Feedback Policies for Motion Skills

SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation

Learning Agile Flight Maneuvers: Deep SE(3) Motion Planning and Control for Quadrotors

ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters

Perpetual Humanoid Control for Real-time Simulated Avatars

Learning and Exploring Motor Skills with Spacetime Bounds

Learning Models For Constraint-Based Motion Parameterization From Interactive Physics-Based Simulation

DReCon