Abstract:This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot's I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.

A Biped Robot Learning to Walk like Human by Reinforcement Learning.

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Realization of a Human-like Gait for a Bipedal Robot Based on Gait Analysis

Teach Biped Robots to Walk via Gait Principles and Reinforcement Learning with Adversarial Critics

Learning Bipedal Walking for Humanoids with Current Feedback

Human-Like Walking with Heel off and Toe Support for Biped Robot

A Heuristics-Based Reinforcement Learning Method to Control Bipedal Robots

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning

Learning Bipedal Robot Locomotion from Human Movement

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control

Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking

Robust biped locomotion using deep reinforcement learning on top of an analytical control approach

Bipedal Walking Robot using Deep Deterministic Policy Gradient

Agile and versatile bipedal robot tracking control through reinforcement learning

Modeling and reinforcement learning-based locomotion control for a humanoid robot with kinematic loop closures

Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

Realizing Human-like Walking and Running with Feedforward Enhanced Reinforcement Learning.

Learning Responsive Humanoid Motion Skills from Graph-Powered Motion Matching

Multi-Speed Walking Gait Generation for Bipedal Robots Based on Reinforcement Learning and Human Motion Imitation