Abstract:Stable gait generation is a crucial problem for legged robot locomotion as this impacts other critical performance factors such as, e.g. mobility over an uneven terrain and power consumption. Gait generation stability results from the efficient control of the interaction between the legged robot's body and the environment where it moves. Here, we study how this can be achieved by a combination of model-predictive and predictive reinforcement learning controllers. Model-predictive control (MPC) is a well-established method that does not utilize any online learning (except for some adaptive variations) as it provides a convenient interface for state constraints management. Reinforcement learning (RL), in contrast, relies on adaptation based on pure experience. In its bare-bone variants, RL is not always suitable for robots due to their high complexity and expensive simulation/experimentation. In this work, we combine both control methods to address the quadrupedal robot stable gate generation problem. The hybrid approach that we develop and apply uses a cost roll-out algorithm with a tail cost in the form of a Q-function modeled by a neural network; this allows to alleviate the computational complexity, which grows exponentially with the prediction horizon in a purely MPC approach. We demonstrate that our RL gait controller achieves stable locomotion at short horizons, where a nominal MP controller fails. Further, our controller is capable of live operation, meaning that it does not require previous training. Our results suggest that the hybridization of MPC with RL, as presented here, is beneficial to achieve a good balance between online control capabilities and computational complexity.

Learning Koopman Dynamics for Safe Legged Locomotion with Reinforcement Learning-based Controller

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Continual Learning and Lifting of Koopman Dynamics for Linear Control of Legged Robots

Safe Learning of Locomotion Skills from MPC

Online Learning of Unknown Dynamics for Model-Based Controllers in Legged Locomotion

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Learning Koopman Operators with Control Using Bi-level Optimization

Koopman Operator-Based Knowledge-Guided Reinforcement Learning for Safe Human-Robot Interaction

Deep Koopman Operator Based Model Predictive Control for Nonlinear Robotics Systems

Learning Forward Dynamics Model and Informed Trajectory Sampler for Safe Quadruped Navigation

PIP-Loco: A Proprioceptive Infinite Horizon Planning Framework for Quadrupedal Robot Locomotion

Multi-Layered Safety for Legged Robots via Control Barrier Functions and Model Predictive Control

Centroidal State Estimation based on the Koopman Embedding for Dynamic Legged Locomotion

A Distributed Deep Koopman Learning Algorithm for Control

Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion

Quadrotor Trajectory Tracking with Learned Dynamics: Joint Koopman-based Learning of System Models and Function Dictionaries

Safety-critical Locomotion of Biped Robots in Infeasible Paths: Overcoming Obstacles during Navigation toward Destination

Safety-critical Motion Planning for Collaborative Legged Loco-Manipulation over Discrete Terrain

Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion

Episodic Koopman Learning of Nonlinear Robot Dynamics with Application to Fast Multirotor Landing

Learning Control Policies for Fall prevention and safety in bipedal locomotion