Abstract:This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot's I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.

Perception-Driven Learning of High-Dynamic Jumping Motions for Single-Legged Robots

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Learning Jumping Skills from Human with a Fast Reinforcement Learning Framework

Continuous Versatile Jumping Using Learned Action Residuals

A Multiobjective Collaborative Deep Reinforcement Learning Algorithm for Jumping Optimization of Bipedal Robot

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Stable Jumping Control Based on Deep Reinforcement Learning for a Locust-Inspired Robot

Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Design

Adaptive Locomotion Control of Sixteen-legged Robot Based on Deep Reinforcement Learning

Learning to Jump from Pixels

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Impedance Matching: Enabling an RL-Based Running Jump in a Quadruped Robot

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control

Bionic Jumping of Humanoid Robot via Online Centroid Trajectory Optimization and High Dynamic Motion Controller

Cat-like Jumping and Landing of Legged Robots in Low-gravity Using Deep Reinforcement Learning

Robust and Versatile Bipedal Jumping Control through Reinforcement Learning

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

Learning Agile, Robust Locomotion Skills for Quadruped Robot.

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning