Abstract:In the context of legged robots, adaptive behavior involves adaptive balancing and adaptive swing foot reflection. While adaptive balancing counteracts perturbations to the robot, adaptive swing foot reflection helps the robot to navigate intricate terrains without foot entrapment. In this paper, we manage to bring both aspects of adaptive behavior to quadruped locomotion by combining RL and MPC while improving the robustness and agility of blind legged locomotion. This integration leverages MPC's strength in predictive capabilities and RL's adeptness in drawing from past experiences. Unlike traditional locomotion controls that separate stance foot control and swing foot trajectory, our innovative approach unifies them, addressing their lack of synchronization. At the heart of our contribution is the synthesis of stance foot control with swing foot reflection, improving agility and robustness in locomotion with adaptive behavior. A hallmark of our approach is robust blind stair climbing through swing foot reflection. Moreover, we intentionally designed the learning module as a general plugin for different robot platforms. We trained the policy and implemented our approach on the Unitree A1 robot, achieving impressive results: a peak turn rate of 8.5 rad/s, a peak running speed of 3 m/s, and steering at a speed of 2.5 m/s. Remarkably, this framework also allows the robot to maintain stable locomotion while bearing an unexpected load of 10 kg, or 83\% of its body mass. We further demonstrate the generalizability and robustness of the same policy where it realizes zero-shot transfer to different robot platforms like Go1 and AlienGo robots for load carrying. Code is made available for the use of the research community at <a class="link-external link-https" href="https://github.com/DRCL-USC/RL_augmented_MPC.git" rel="external noopener nofollow">this https URL</a>

Manipulator as a Tail: Promoting Dynamic Stability for Legged Locomotion

Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Learning Rapid Turning, Aerial Reorientation, and Balancing using Manipulator as a Tail

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Manipulability Enhancement of Legged Manipulators by Adaptive Motion Distribution

Visual Manipulation with Legs

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Learning Visual Quadrupedal Loco-Manipulation from Demonstrations

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Learning Force Control for Legged Manipulation

Pedipulate: Enabling Manipulation Skills using a Quadruped Robot's Leg

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion

Adaptive legged manipulation: Versatile disturbance predictive control for quadruped robots with robotic arms

A Model Predictive Approach for Online Mobile Manipulation of Nonholonomic Objects using Learned Dynamics

Arm-Constrained Curriculum Learning for Loco-Manipulation of the Wheel-Legged Robot

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Bridging Locomotion and Manipulation Using Reconfigurable Robotic Limbs via Reinforcement Learning

Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control