Abstract:This paper addresses the challenge of terrain-adaptive dynamic locomotion in humanoid robots, a problem traditionally tackled by optimization-based methods or reinforcement learning (RL). Optimization-based methods, such as model-predictive control, excel in finding optimal reaction forces and achieving agile locomotion, especially in quadruped, but struggle with the nonlinear hybrid dynamics of legged systems and the real-time computation of step location, timing, and reaction forces. Conversely, RL-based methods show promise in navigating dynamic and rough terrains but are limited by their extensive data requirements. We introduce a novel locomotion architecture that integrates a neural network policy, trained through RL in simplified environments, with a state-of-the-art motion controller combining model-predictive control (MPC) and whole-body impulse control (WBIC). The policy efficiently learns high-level locomotion strategies, such as gait selection and step positioning, without the need for full dynamics simulations. This control architecture enables humanoid robots to dynamically navigate discrete terrains, making strategic locomotion decisions (e.g., walking, jumping, and leaping) based on ground height maps. Our results demonstrate that this integrated control architecture achieves dynamic locomotion with significantly fewer training samples than conventional RL-based methods and can be transferred to different humanoid platforms without additional training. The control architecture has been extensively tested in dynamic simulations, accomplishing terrain height-based dynamic locomotion for three different robots.

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains

Hybrid Bipedal Locomotion Based on Reinforcement Learning and Heuristics

Adaptive Gait Acquisition through Learning Dynamic Stimulus Instinct of Bipedal Robot

Humanoid Adaptive Locomotion Control Through a Bioinspired CPG-based Controller

MorAL: Learning Morphologically Adaptive Locomotion Controller for Quadrupedal Robots on Challenging Terrains

Adapting Rapid Motor Adaptation for Bipedal Robots

Learning Agile Bipedal Motions on a Quadrupedal Robot

Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Efficient Dynamic Locomotion of Quadruped Robot Via Adaptive Diagonal Gait

Model Predictive Control for Motion Planning of Quadrupedal Locomotion.

Perceptive Locomotion of Legged Robot Coupling Model Predictive Control and Terrain Mapping

Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller

Learning Bipedal Walking on a Quadruped Robot via Adversarial Motion Priors

Learning Bipedal Walking for Humanoids with Current Feedback

Modeling and reinforcement learning-based locomotion control for a humanoid robot with kinematic loop closures

Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animals

An Adaptive Feedforward Control Method for Under-Actuated Bipedal Walking on the Compliant Ground

Agile and versatile bipedal robot tracking control through reinforcement learning