Abstract:Legged locomotion is a complex control problem that requires both accuracy and robustness to cope with real-world challenges. Legged systems have traditionally been controlled using trajectory optimization with inverse dynamics. Such hierarchical model-based methods are appealing due to intuitive cost function tuning, accurate planning, generalization, and most importantly, the insightful understanding gained from more than one decade of extensive research. However, model mismatch and violation of assumptions are common sources of faulty operation. Simulation-based reinforcement learning, on the other hand, results in locomotion policies with unprecedented robustness and recovery skills. Yet, all learning algorithms struggle with sparse rewards emerging from environments where valid footholds are rare, such as gaps or stepping stones. In this work, we propose a hybrid control architecture that combines the advantages of both worlds to simultaneously achieve greater robustness, foot-placement accuracy, and terrain generalization. Our approach utilizes a model-based planner to roll out a reference motion during training. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We evaluate the accuracy of our locomotion pipeline on sparse terrains, where pure data-driven methods are prone to fail. Furthermore, we demonstrate superior robustness in the presence of slippery or deformable ground when compared to model-based counterparts. Finally, we show that our proposed tracking controller generalizes across different trajectory optimization methods not seen during training. In conclusion, our work unites the predictive capabilities and optimality guarantees of online planning with the inherent robustness attributed to offline learning.

Non-Gaited Legged Locomotion with Monte-Carlo Tree Search and Supervised Learning

Monte Carlo Tree Search Gait Planner for Non-Gaited Legged System Control

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Perceptive Locomotion of Legged Robot Coupling Model Predictive Control and Terrain Mapping

Diffusion-based learning of contact plans for agile locomotion

Online Optimization of Central Pattern Generators for Quadruped Locomotion

Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion

Quadrupedal Footstep Planning using Learned Motion Models of a Black-Box Controller

Perceptive Locomotion through Nonlinear Model Predictive Control

Online Dynamic Trajectory Optimization and Control for a Quadruped Robot

RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

Combined Sampling and Optimization Based Planning for Legged-Wheeled Robots

Fault Tolerant Free Gait and Footstep Planning for Hexapod Robot Based on Monte-Carlo Tree

Locomotion Planning through a Hybrid Bayesian Trajectory Optimization

Optimization-Based Control for Dynamic Legged Robots

Contact Sequence Planning for Hexapod Robots in Sparse Foothold Environment Based on Monte-Carlo Tree

Navigation for Legged Mobility: Dynamic Climbing

LSTP: Long Short-Term Motion Planning for Legged and Legged-Wheeled Systems

Contact-conditioned learning of locomotion policies

DTC: Deep Tracking Control