Abstract:In the context of legged robots, adaptive behavior involves adaptive balancing and adaptive swing foot reflection. While adaptive balancing counteracts perturbations to the robot, adaptive swing foot reflection helps the robot to navigate intricate terrains without foot entrapment. In this paper, we manage to bring both aspects of adaptive behavior to quadruped locomotion by combining RL and MPC while improving the robustness and agility of blind legged locomotion. This integration leverages MPC's strength in predictive capabilities and RL's adeptness in drawing from past experiences. Unlike traditional locomotion controls that separate stance foot control and swing foot trajectory, our innovative approach unifies them, addressing their lack of synchronization. At the heart of our contribution is the synthesis of stance foot control with swing foot reflection, improving agility and robustness in locomotion with adaptive behavior. A hallmark of our approach is robust blind stair climbing through swing foot reflection. Moreover, we intentionally designed the learning module as a general plugin for different robot platforms. We trained the policy and implemented our approach on the Unitree A1 robot, achieving impressive results: a peak turn rate of 8.5 rad/s, a peak running speed of 3 m/s, and steering at a speed of 2.5 m/s. Remarkably, this framework also allows the robot to maintain stable locomotion while bearing an unexpected load of 10 kg, or 83\% of its body mass. We further demonstrate the generalizability and robustness of the same policy where it realizes zero-shot transfer to different robot platforms like Go1 and AlienGo robots for load carrying. Code is made available for the use of the research community at <a class="link-external link-https" href="https://github.com/DRCL-USC/RL_augmented_MPC.git" rel="external noopener nofollow">this https URL</a>

Solving the swing-up and balance task for the Acrobot and Pendubot with SAC

Average-Reward Maximum Entropy Reinforcement Learning for Underactuated Double Pendulum Tasks

Combining Deep Reinforcement Learning And Local Control For The Acrobot Swing-up And Balance Task

Velocity-History-Based Soft Actor-Critic Tackling IROS'24 Competition "AI Olympics with RealAIGym"

A Deep Reinforcement Learning Control Method for a Four-Link Brachiation Robot

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

A Deep Reinforcement Learning Approach towards Pendulum Swing-up Problem based on TF-Agents

Acrobot Swing Up with MATLAB

Intelligent Control for an Acrobot

Experimental Study on Reinforcement Learning-based Control of an Acrobot

Learning control of underactuated double pendulum with Model-Based Reinforcement Learning

A Deep Reinforcement Learning Solution for the Low Level Motion Control of a Robot Manipulator System

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learning

Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraint

SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Tasks

A Strategy-Oriented Bayesian Soft Actor-Critic Model

Robotic Manipulator in Dynamic Environment with SAC Combing Attention Mechanism and LSTM

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

A Q-learning approach to the continuous control problem of robot inverted pendulum balancing

Autonomous Control of a Novel Closed Chain Five Bar Active Suspension via Deep Reinforcement Learning