Abstract:The agility of animals, particularly in complex activities such as running, turning, jumping, and backflipping, stands as an exemplar for robotic system design. Transferring this suite of behaviors to legged robotic systems introduces essential inquiries: How can a robot be trained to learn multiple locomotion behaviors simultaneously? How can the robot execute these tasks with a smooth transition? How to integrate these skills for wide-range applications? This paper introduces the Versatile Instructable Motion prior (VIM) - a Reinforcement Learning framework designed to incorporate a range of agile locomotion tasks suitable for advanced robotic applications. Our framework enables legged robots to learn diverse agile low-level skills by imitating animal motions and manually designed motions. Our Functionality reward guides the robot's ability to adopt varied skills, and our Stylization reward ensures that robot motions align with reference motions. Our evaluations of the VIM framework span both simulation environments and real-world deployment. To the best of our knowledge, this is the first work that allows a robot to concurrently learn diverse agile locomotion skills using a single learning-based controller in the real world. Further details and supportive media can be found at our project site: <a class="link-external link-https" href="https://rchalyang.github.io/VIM" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to enable robots to learn multiple agile movement skills simultaneously and achieve a smooth transition when performing these tasks, so as to be applicable to a wide range of application scenarios. Specifically, the main problems faced by the researchers are: 1. **How to train robots to learn multiple movement behaviors simultaneously**: For example, complex activities such as running, turning, jumping and backflips. 2. **How to enable robots to smoothly transition between these tasks**: Seamlessly switch between different movement skills to adapt to different environmental and task requirements. 3. **How to integrate these skills for application in a wide range of scenarios**: Enable robots to exhibit agility comparable to that of natural quadruped animals in a variety of complex environments. To address these problems, the paper proposes a reinforcement - learning framework named "Versatile Instructable Motion prior (VIM)". This framework aims to enable legged robots to learn multiple agile low - level skills by imitating animal movements and manually - designed movements. The VIM framework achieves this goal through the following two main mechanisms: - **Functionality Reward**: Guide the robot to adopt different skills and ensure that the robot can complete specific task objectives, such as moving forward at a predetermined speed. - **Stylization Reward**: Ensure that the robot's movements are consistent with the reference movements, so that not only the functional goal is achieved, but also the form of movement is close to natural or designed movements. Through these mechanisms, the VIM framework can be evaluated in both simulated environments and the real world, and, to the best of the authors' knowledge, this is the first work that can use a single learning - based controller to learn multiple agile movement skills simultaneously in the real world.

Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior

Motion Control of Bionic Robots Via Biomimetic Learning

A GIM-based approach for biomimetic robot motion learning.

FastMimic: Model-Based Motion Imitation for Agile, Diverse and Generalizable Quadrupedal Locomotion

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

Imitation and Adaptation Based on Consistency: A Quadruped Robot Imitates Animals from Videos Using Deep Reinforcement Learning

Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animals

Learning Agile Bipedal Motions on a Quadrupedal Robot

Learning agility and adaptive legged locomotion via curricular hindsight reinforcement learning

Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response

An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Learning Responsive Humanoid Motion Skills from Graph-Powered Motion Matching

Learning Agile Locomotion Skills with a Mentor

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control