Abstract:Stable gait generation is a crucial problem for legged robot locomotion as this impacts other critical performance factors such as, e.g. mobility over an uneven terrain and power consumption. Gait generation stability results from the efficient control of the interaction between the legged robot's body and the environment where it moves. Here, we study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. Model-predictive control (MPC) is a well-established method that does not utilize any online learning (except for some adaptive variations) as it provides a convenient interface for state constraints management. Reinforcement learning (RL), in contrast, relies on adaptation based on pure experience. In its bare-bone variants, RL is not always suitable for robots due to their high complexity and expensive simulation/experimentation. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem. The hybrid approach that we develop and apply uses a cost roll-out algorithm with a tail cost in the form of a Q-function modeled by a neural network; this allows to alleviate the computational complexity, which grows exponentially with the prediction horizon in a purely MPC approach. We demonstrate that our RL gait controller achieves stable locomotion at short horizons, where a nominal MP controller fails. Further, our controller is capable of live operation, meaning that it does not require previous training. Our results suggest that the hybridization of MPC with RL, as presented here, is beneficial to achieve a good balance between online control capabilities and computational complexity.

MPC for Humanoid Gait Generation: Stability and Feasibility

Nonlinear MPC-Based Control Framework for Quadruped Robots: Touch-Down in Complex Terrain

Discrete time model predictive control for humanoid walking with step adjustment

Fast Online Planning for Bipedal Locomotion Via Centroidal Model Predictive Gait Synthesis

Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion

Adapting Gait Frequency for Posture-regulating Humanoid Push-recovery via Hierarchical Model Predictive Control

Nonlinear model predictive control for robust bipedal locomotion: exploring angular momentum and CoM height changes

Whole-body MPC for highly redundant legged manipulators: experimental evaluation with a 37 DoF dual-arm quadruped

Robust Locomotion Exploiting Multiple Balance Strategies: An Observer-Based Cascaded Model Predictive Control Approach

Perceptive Locomotion through Whole-Body MPC and Optimal Region Selection

A Hierarchical MPC for End-Effector Tracking Control of Legged Mobile Manipulators

Online DNN-driven Nonlinear MPC for Stylistic Humanoid Robot Walking with Step Adjustment

Bipedal Walking on Constrained Footholds with MPC Footstep Control

Variable Inertia Model Predictive Control for Fast Bipedal Maneuvers

A Model Predictive Capture Point Control Framework for Robust Humanoid Balancing via Ankle, Hip, and Stepping Strategies

A Collision-Free MPC for Whole-Body Dynamic Locomotion and Manipulation

Vision Assisted Control of Lower Extremity Exoskeleton for Obstacle Avoidance With Dynamic Constraint Based Piecewise Nonlinear MPC

Legged Robot Locomotion Via Hybrid Zero Dynamics and Model Predictive Control

Contingency Model Predictive Control for Bipedal Locomotion on Moving Surfaces with a Linear Inverted Pendulum Model

Contact-Implicit MPC: Controlling Diverse Quadruped Motions Without Pre-Planned Contact Modes or Trajectories

Variable Horizon MPC With Swing Foot Dynamics for Bipedal Walking Control