Abstract:The International Journal of Robotics Research, Ahead of Print. This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot's I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances. The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.

Single Leg Operational Space Control of Quadruped Robot Based on Reinforcement Learning

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Reinforcement Learning based Control of a Quadruped Robot

Meta-Reinforcement Learning of Hierarchical Fault-Tolerant Controller for Multiple Leg Failures in Hexapod Robots

Model Predictive Control of Quadruped Robot Based on Reinforcement Learning

A Motion Planning and Control Method of Quadruped Robot Based on Deep Reinforcement Learning

High Dynamic Position Control for a Typical Hydraulic Quadruped Robot Leg Based on Virtual Decomposition Control

A Heuristics-Based Reinforcement Learning Method to Control Bipedal Robots

Robust Control of Quadruped Robots Using Reinforcement Learning and Depth Completion Network

Learning Agile, Robust Locomotion Skills for Quadruped Robot.

Learning Smooth and Omnidirectional Locomotion for Quadruped Robots

Animal-Like Eye Vision Assisted Locomotion of a Quadruped Based on Reinforcement Learning.

Position/Force Control For A Single Leg Of A Quadruped Robot In An Operation Space

Reinforcement Learning For Quadrupedal Locomotion: Current Advancements And Future Perspectives

Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning

An advanced reinforcement learning control method for quadruped robots in typical urban terrains

Environmental Features Assessment Network Aided Deep Reinforcement Learning for Quadrupedal Locomotion in Tough Terrain

Simultaneous locomotion and manipulation control of quadruped robots using reinforcement learning-based adaptive fractional-order sliding-mode control

Learning Fast and Agile Quadrupedal Locomotion over Complex Terrain

Agile Control for Quadruped Robot in Complex Environment Based on Deep Reinforcement Learning Method.

Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control