Abstract:This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot's I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.

Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Models

Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Models

Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models

Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads

Demonstrating A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning

Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

Learning Task Space Actions for Bipedal Locomotion

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Reinforcement Learning Controllers for Soft Robots using Learned Environments

SLR: Learning Quadruped Locomotion without Privileged Information

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control

SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Rendering

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfer

Online Learning of Unknown Dynamics for Model-Based Controllers in Legged Locomotion

Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments

Training in Task Space to Speed Up and Guide Reinforcement Learning

Robust Walking and Sim-to-Real Optimization for Quadruped Robots via Reinforcement Learning

Learning and Adapting Agile Locomotion Skills by Transferring Experience