Abstract:Stable gait generation is a crucial problem for legged robot locomotion as this impacts other critical performance factors such as, e.g. mobility over an uneven terrain and power consumption. Gait generation stability results from the efficient control of the interaction between the legged robot's body and the environment where it moves. Here, we study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. Model-predictive control (MPC) is a well-established method that does not utilize any online learning (except for some adaptive variations) as it provides a convenient interface for state constraints management. Reinforcement learning (RL), in contrast, relies on adaptation based on pure experience. In its bare-bone variants, RL is not always suitable for robots due to their high complexity and expensive simulation/experimentation. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem. The hybrid approach that we develop and apply uses a cost roll-out algorithm with a tail cost in the form of a Q-function modeled by a neural network; this allows to alleviate the computational complexity, which grows exponentially with the prediction horizon in a purely MPC approach. We demonstrate that our RL gait controller achieves stable locomotion at short horizons, where a nominal MP controller fails. Further, our controller is capable of live operation, meaning that it does not require previous training. Our results suggest that the hybridization of MPC with RL, as presented here, is beneficial to achieve a good balance between online control capabilities and computational complexity.

Bilevel Optimization for Real-Time Control with Application to Locomotion Gait Generation

Nonlinear MPC-Based Control Framework for Quadruped Robots: Touch-Down in Complex Terrain

Real-Time Control of Full Actuated Biped Robot Based on Nonlinear Model Predictive Control

Optimal Control for Quadruped Locomotion using LTV MPC

Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing

Real-time Walking Pattern Optimization for Humanoid Robot Based on Model Predictive Control

Legged Robot Locomotion Via Hybrid Zero Dynamics and Model Predictive Control

Learning Near-global-optimal Strategies for Hybrid Non-convex Model Predictive Control of Single Rigid Body Locomotion

Motion Planning for a Bounding Quadruped Robot Using Ilqg Based MPC

Inverse-Dynamics MPC via Nullspace Resolution

A Hierarchical MPC for End-Effector Tracking Control of Legged Mobile Manipulators

Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control

Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion

Reduced Model Predictive Control Toward Highly Dynamic Quadruped Locomotion

Robust Locomotion Exploiting Multiple Balance Strategies: An Observer-Based Cascaded Model Predictive Control Approach

Dynamic Legged Manipulation of a Ball Through Multi-Contact Optimization

Accelerating Model Predictive Control for Legged Robots through Distributed Optimization

Optimization-Based Control for Dynamic Legged Robots

Real-Time Whole-Body Control of Legged Robots with Model-Predictive Path Integral Control

Bounding gait control of a parallel quadruped robot

Fast Contact-Implicit Model Predictive Control