Abstract:Navigating rugged landscapes poses significant challenges for legged locomotion. Multi-legged robots (those with 6 and greater) offer a promising solution for such terrains, largely due to their inherent high static stability, resulting from a low center of mass and wide base of support. Such systems require minimal effort to maintain balance. Recent studies have shown that a linear controller, which modulates the vertical body undulation of a multi-legged robot in response to shifts in terrain roughness, can ensure reliable mobility on challenging terrains. However, the potential of a learning-based control framework that adjusts multiple parameters to address terrain heterogeneity remains underexplored. We posit that the development of an experimentally validated physics-based simulator for this robot can rapidly advance capabilities by allowing wide parameter space exploration. Here we develop a MuJoCo-based simulator tailored to this robotic platform and use the simulation to develop a reinforcement learning-based control framework that dynamically adjusts horizontal and vertical body undulation, and limb stepping in real-time. Our approach improves robot performance in simulation, laboratory experiments, and outdoor tests. Notably, our real-world experiments reveal that the learning-based controller achieves a 30\% to 50\% increase in speed compared to a linear controller, which only modulates vertical body waves. We hypothesize that the superior performance of the learning-based controller arises from its ability to adjust multiple parameters simultaneously, including limb stepping, horizontal body wave, and vertical body wave.

Deep Reinforcement Learning-Based Control of Bicycle Robots on Rough Terrain

Adaptive and Robust Terrain Classification Control Algorithm for a Spherical Robot

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Meta-Reinforcement Learning of Hierarchical Fault-Tolerant Controller for Multiple Leg Failures in Hexapod Robots

Online Series-Parallel Reinforcement-Learning-Based Balancing Control for Reaction Wheel Bicycle Robots on a Curved Pavement

High Maneuverability Control of Single-track Two-wheeled Robot in Narrow Terrain Based on Reinforcement Learning.

Natural Residual Reinforcement Learning for Bicycle Robot Control

Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain

Online Reinforcement-Learning-Based Adaptive Terminal Sliding Mode Control for Disturbed Bicycle Robots on a Curved Pavement

Composite Deep Learning Control for Autonomous Bicycles by Using Deep Deterministic Policy Gradient

Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

An advanced reinforcement learning control method for quadruped robots in typical urban terrains

Combined control algorithm based on synchronous reinforcement learning for a self-balancing bicycle robot

Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

Learning-Based Trajectory Tracking and Balance Control for Bicycle Robots with a Pendulum: A Gaussian Process Approach

Learning to enhance multi-legged robot on rugged landscapes

Intelligent Balancing and Trajectory Tracking Control for Unicycle Robots

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control