Abstract:Stable gait generation is a crucial problem for legged robot locomotion as this impacts other critical performance factors such as, e.g. mobility over an uneven terrain and power consumption. Gait generation stability results from the efficient control of the interaction between the legged robot's body and the environment where it moves. Here, we study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. Model-predictive control (MPC) is a well-established method that does not utilize any online learning (except for some adaptive variations) as it provides a convenient interface for state constraints management. Reinforcement learning (RL), in contrast, relies on adaptation based on pure experience. In its bare-bone variants, RL is not always suitable for robots due to their high complexity and expensive simulation/experimentation. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem. The hybrid approach that we develop and apply uses a cost roll-out algorithm with a tail cost in the form of a Q-function modeled by a neural network; this allows to alleviate the computational complexity, which grows exponentially with the prediction horizon in a purely MPC approach. We demonstrate that our RL gait controller achieves stable locomotion at short horizons, where a nominal MP controller fails. Further, our controller is capable of live operation, meaning that it does not require previous training. Our results suggest that the hybridization of MPC with RL, as presented here, is beneficial to achieve a good balance between online control capabilities and computational complexity.

Controlled Recurrence of a Biped with Torso

Stable running of a planar underactuated biped robot

Exponentially Stabilizing Continuous-Time Controllers for multi-domain hybrid systems with application to 3D bipdeal walking.

A Stochastic Policy Gradient Based Adaptive Control For Biped Walking

A Sustainable Running Criterion for Biped Balance Control

Local stability of PD controlled bipedal walking robots

Time-scaling control and passive walking of bipeds with underactuation degree one

Dynamically Stable 3D Quadrupedal Walking with Multi-Domain Hybrid System Models and Virtual Constraint Controllers

Bifurcation Gait Suppression of a Bipedal Walking Robot with a Torso Based on Model Predictive Control

Safe Whole-Body Task Space Control for Humanoid Robots

Multi-Layered Safety for Legged Robots via Control Barrier Functions and Model Predictive Control

Provably Stabilizing Global-Position Tracking Control for Hybrid Models of Multi-Domain Bipedal Walking via Multiple Lyapunov Analysis

Exponential Stabilization of Fully Actuated Planar Bipedal Robotic Walking with Global Position Tracking Capabilities

Design and Control of a Multi-Locomotion Parallel-Legged Bipedal Robot

Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion

Legged Robot Locomotion Via Hybrid Zero Dynamics and Model Predictive Control

Adaptive Time-Delay Balance Control of Biped Robots.

Terrain Adaptive Gait Transitioning for a Quadruped Robot using Model Predictive Control

Singular Configurations and Underactuated Balance Control for Biped Robot with Point Contact

Real-Time Control of Full Actuated Biped Robot Based on Nonlinear Model Predictive Control

Human-Like Walking with Heel off and Toe Support for Biped Robot