Abstract:Recent advances of locomotion controllers utilizing deep reinforcement learning (RL) have yielded impressive results in terms of achieving rapid and robust locomotion across challenging terrain, such as rugged rocks, non-rigid ground, and slippery surfaces. However, while these controllers primarily address challenges underneath the robot, relatively little research has investigated legged mobility through confined 3D spaces, such as narrow tunnels or irregular voids, which impose all-around constraints. The cyclic gait patterns resulted from existing RL-based methods to learn parameterized locomotion skills characterized by motion parameters, such as velocity and body height, may not be adequate to navigate robots through challenging confined 3D spaces, requiring both agile 3D obstacle avoidance and robust legged locomotion. Instead, we propose to learn locomotion skills end-to-end from goal-oriented navigation in confined 3D spaces. To address the inefficiency of tracking distant navigation goals, we introduce a hierarchical locomotion controller that combines a classical planner tasked with planning waypoints to reach a faraway global goal location, and an RL-based policy trained to follow these waypoints by generating low-level motion commands. This approach allows the policy to explore its own locomotion skills within the entire solution space and facilitates smooth transitions between local goals, enabling long-term navigation towards distant goals. In simulation, our hierarchical approach succeeds at navigating through demanding confined 3D environments, outperforming both pure end-to-end learning approaches and parameterized locomotion skills. We further demonstrate the successful real-world deployment of our simulation-trained controller on a real robot.

Hierarchical RL-Guided Large-scale Navigation of a Snake Robot

Reinforcement Learning of Serpentine Locomotion for a Snake Robot

Snake Robot with Tactile Perception Navigates on Large-scale Challenging Terrain

Reinforcement learning-based motion control for snake robots in complex environments

Steering Motion Control of a Snake Robot Via a Biomimetic Approach

Reinforcement Learning of CPG-regulated Locomotion Controller for a Soft Snake Robot

A Reinforcement Learning-Based Strategy of Path Following for Snake Robots with an Onboard Camera

Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning

Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot

Autonomous, Monocular, Vision-Based Snake Robot Navigation and Traversal of Cluttered Environments using Rectilinear Gait Motion

Motion Planning and Iterative Learning Control of a Modular Soft Robotic Snake

Towards Autonomous Locomotion: CPG-based Control of Smooth 3D Slithering Gait Transition of a Snake-Like Robot.

Motion Control Of A Snake Robot Via Cerebellum-Inspired Learning Control

Bio-Inspired Multimodal Motion Gait Control of Snake Robots with Environmental Adaptability Based on ROS

Adaptive Path Following and Locomotion Optimization of Snake-Like Robot Controlled by the Central Pattern Generator

An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep Reinforcement Learning

A Selftuning Multiphase CPG Enabling the Snake Ro Bot to Adapt to Environments

Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Representation Reinforcement Learning-Based Dense Control for Point Following with State Sparse Sensing of 3-D Snake Robots

Directional Compliance in Obstacle-Aided Navigation for Snake Robots