Abstract:In the context of legged robots, adaptive behavior involves adaptive balancing and adaptive swing foot reflection. While adaptive balancing counteracts perturbations to the robot, adaptive swing foot reflection helps the robot to navigate intricate terrains without foot entrapment. In this paper, we manage to bring both aspects of adaptive behavior to quadruped locomotion by combining RL and MPC while improving the robustness and agility of blind legged locomotion. This integration leverages MPC's strength in predictive capabilities and RL's adeptness in drawing from past experiences. Unlike traditional locomotion controls that separate stance foot control and swing foot trajectory, our innovative approach unifies them, addressing their lack of synchronization. At the heart of our contribution is the synthesis of stance foot control with swing foot reflection, improving agility and robustness in locomotion with adaptive behavior. A hallmark of our approach is robust blind stair climbing through swing foot reflection. Moreover, we intentionally designed the learning module as a general plugin for different robot platforms. We trained the policy and implemented our approach on the Unitree A1 robot, achieving impressive results: a peak turn rate of 8.5 rad/s, a peak running speed of 3 m/s, and steering at a speed of 2.5 m/s. Remarkably, this framework also allows the robot to maintain stable locomotion while bearing an unexpected load of 10 kg, or 83\% of its body mass. We further demonstrate the generalizability and robustness of the same policy where it realizes zero-shot transfer to different robot platforms like Go1 and AlienGo robots for load carrying. Code is made available for the use of the research community at <a class="link-external link-https" href="https://github.com/DRCL-USC/RL_augmented_MPC.git" rel="external noopener nofollow">this https URL</a>

Learning Push Recovery Behaviors for Humanoid Walking Using Deep Reinforcement Learning

Learning Push Recovery for A Bipedal Humanoid Robot with Dynamical Movement Primitives

On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning

Push Recovery for the Standing Under-Actuated Bipedal Robot Using the Hip Strategy

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

Learning Humanoid Robot Running Skills through Proximal Policy Optimization

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Hybrid Autonomous Controller for Bipedal Robot Balance with Deep Reinforcement Learning and Pattern Generators

Learning Whole-body Motor Skills for Humanoids

Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region

Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning

Anti-push Method of Biped Robot Based on Motion Capture Point and Reinforcement Learning

Robust Recovery Motion Control for Quadrupedal Robots via Learned Terrain Imagination

Robust biped locomotion using deep reinforcement learning on top of an analytical control approach

Learning Control Policies for Fall prevention and safety in bipedal locomotion

Learning Agile Locomotion on Risky Terrains

Data Driven Computational Model for Bipedal Walking and Push Recovery

Push Recovery Controller For Bipedal Robot Walking

Learning Bipedal Walking for Humanoid Robots in Challenging Environments with Obstacle Avoidance

Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC