Abstract:Recent advances of locomotion controllers utilizing deep reinforcement learning (RL) have yielded impressive results in terms of achieving rapid and robust locomotion across challenging terrain, such as rugged rocks, non-rigid ground, and slippery surfaces. However, while these controllers primarily address challenges underneath the robot, relatively little research has investigated legged mobility through confined 3D spaces, such as narrow tunnels or irregular voids, which impose all-around constraints. The cyclic gait patterns resulted from existing RL-based methods to learn parameterized locomotion skills characterized by motion parameters, such as velocity and body height, may not be adequate to navigate robots through challenging confined 3D spaces, requiring both agile 3D obstacle avoidance and robust legged locomotion. Instead, we propose to learn locomotion skills end-to-end from goal-oriented navigation in confined 3D spaces. To address the inefficiency of tracking distant navigation goals, we introduce a hierarchical locomotion controller that combines a classical planner tasked with planning waypoints to reach a faraway global goal location, and an RL-based policy trained to follow these waypoints by generating low-level motion commands. This approach allows the policy to explore its own locomotion skills within the entire solution space and facilitates smooth transitions between local goals, enabling long-term navigation towards distant goals. In simulation, our hierarchical approach succeeds at navigating through demanding confined 3D environments, outperforming both pure end-to-end learning approaches and parameterized locomotion skills. We further demonstrate the successful real-world deployment of our simulation-trained controller on a real robot.

Dynamic terrain traversal skills using reinforcement learning

Stability and Agility: Biped Running over Varied and Unknown Terrain

Terrain runner: control, parameterization, composition, and planning for highly dynamic motions

Terrain Runner

Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains

Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning

Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning

Walking with Terrain Reconstruction: Learning to Traverse Risky Sparse Footholds

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control

Learning Vision-Based Bipedal Locomotion for Challenging Terrain

Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning

Learning Setup Policies: Reliable Transition Between Locomotion Behaviours

Terrain-adaptive Central Pattern Generators with Reinforcement Learning for Hexapod Locomotion

RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

Learning Quadrupedal Locomotion on Tough Terrain Using an Asymmetric Terrain Feature Mining Network

Quadruped Robot Locomotion in Unknown Terrain Using Deep Reinforcement Learning

Learning Whole-body Motor Skills for Humanoids

Combined Reinforcement Learning and CPG Algorithm to Generate Terrain-Adaptive Gait of Hexapod Robots