Abstract:Recent advances of locomotion controllers utilizing deep reinforcement learning (RL) have yielded impressive results in terms of achieving rapid and robust locomotion across challenging terrain, such as rugged rocks, non-rigid ground, and slippery surfaces. However, while these controllers primarily address challenges underneath the robot, relatively little research has investigated legged mobility through confined 3D spaces, such as narrow tunnels or irregular voids, which impose all-around constraints. The cyclic gait patterns resulted from existing RL-based methods to learn parameterized locomotion skills characterized by motion parameters, such as velocity and body height, may not be adequate to navigate robots through challenging confined 3D spaces, requiring both agile 3D obstacle avoidance and robust legged locomotion. Instead, we propose to learn locomotion skills end-to-end from goal-oriented navigation in confined 3D spaces. To address the inefficiency of tracking distant navigation goals, we introduce a hierarchical locomotion controller that combines a classical planner tasked with planning waypoints to reach a faraway global goal location, and an RL-based policy trained to follow these waypoints by generating low-level motion commands. This approach allows the policy to explore its own locomotion skills within the entire solution space and facilitates smooth transitions between local goals, enabling long-term navigation towards distant goals. In simulation, our hierarchical approach succeeds at navigating through demanding confined 3D environments, outperforming both pure end-to-end learning approaches and parameterized locomotion skills. We further demonstrate the successful real-world deployment of our simulation-trained controller on a real robot.

Learning Locomotion for Quadruped Robots via Distributional Ensemble Actor-Critic

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Robust Quadrupedal Locomotion Via Risk-Averse Policy Learning

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning

Motion Simulation of Flying Quadruped Robot Based on Deep Reinforcement Learning

Estimating Probability Distribution with Q-learning for Biped Gait Generation and Optimization.

Real-time local path planning strategy based on deep distributional reinforcement learning

Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errors

ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Learning Agile Locomotion on Risky Terrains

Learning a Distributed Hierarchical Locomotion Controller for Embodied Cooperation

PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robots

Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning

Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

Hybrid LMC: Hybrid Learning and Model-based Control for Wheeled Humanoid Robot via Ensemble Deep Reinforcement Learning

Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks