Abstract:Intelligent control of soft robots is challenging due to the nonlinear and difficult-to-model dynamics. One promising model-free approach for soft robot control is reinforcement learning (RL). However, model-free RL methods tend to be computationally expensive and data-inefficient and may not yield natural and smooth locomotion patterns for soft robots. In this work, we develop a bio-inspired design of a learning-based goal-tracking controller for a soft snake robot. The controller is composed of two modules: An RL module for learning goal-tracking behaviors given the unmodeled and stochastic dynamics of the robot, and a central pattern generator (CPG) with the Matsuoka oscillators for generating stable and diverse locomotion patterns. We theoretically investigate the maneuverability of Matsuoka CPG's oscillation bias, frequency, and amplitude for steering control, velocity control, and sim-to-real adaptation of the soft snake robot. Based on this analysis, we proposed a composition of RL and CPG modules such that the RL module regulates the tonic inputs to the CPG system given state feedback from the robot, and the output of the CPG module is then transformed into pressure inputs to pneumatic actuators of the soft snake robot. This design allows the RL agent to naturally learn to entrain the desired locomotion patterns determined by the CPG maneuverability. We validated the optimality and robustness of the control design in both simulation and real experiments, and performed extensive comparisons with state-of-art RL methods to demonstrate the benefit of our bio-inspired control design.

Reinforcement Learning-Based Model Matching in COBRA, a Slithering Snake Robot

Reinforcement Learning-Based Model Matching to Reduce the Sim-Real Gap in COBRA

Reinforcement Learning of Serpentine Locomotion for a Snake Robot

Motion Control Of A Snake Robot Via Cerebellum-Inspired Learning Control

Validation of Tumbling Robot Dynamics with Posture Manipulation for Closed-Loop Heading Angle Control

Perception-Action Coupling Target Tracking Control for a Snake Robot via Reinforcement Learning

Steering Motion Control of a Snake Robot Via a Biomimetic Approach

Reinforcement learning-based motion control for snake robots in complex environments

Loco-Manipulation with Nonimpulsive Contact-Implicit Planning in a Slithering Robot

Dynamic Modeling and Validation of Soft Robotic Snake Locomotion

Motion Planning and Iterative Learning Control of a Modular Soft Robotic Snake

A Thesis on Loco-Manipulation with Non-impulsive Contact-Implicit Planning in a Slithering Robot

Dual-Loop Control of Cable-Driven Snake-like Robots

Machine Learning-Driven Burrowing with a Snake-Like Robot

Reinforcement Learning of CPG-regulated Locomotion Controller for a Soft Snake Robot

Non-impulsive Contact-Implicit Motion Planning for Morpho-functional Loco-manipulation

COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration

Hierarchical RL-Guided Large-scale Navigation of a Snake Robot

Soft Robots Learn to Crawl: Jointly Optimizing Design and Control with Sim-to-Real Transfer

Learning to Locomote with Deep Neural-Network and CPG-based Control in a Soft Snake Robot

Collision Avoidance Regulation-Compliant Orientation Guidance and Maneuvering Control Approach for Snake Robots