Abstract:Bio-inspired Autonomous Underwater Vehicles with soft bodies provide significant performance benefits over conventional propeller-driven vehicles; however, it is difficult to control these vehicles due to their soft underactuated bodies. This study investigates the application of Physical Reservoir Computing (PRC) in the swimmer's flexible body to perform state estimation. This PRC informed state estimation has potential to be used in vehicle control. PRC is a type of recurrent neural network that leverages the nonlinear dynamics of a physical system to predict a nonlinear spatiotemporal input-output relationship. By embodying the neural network into the physical structure, PRC can process the response to an environment input with high computational efficiency. This study uses a soft bio-inspired propulsor embodied as a physical reservoir. We evaluate its ability to predict different state estimation tasks including hydrodynamic forces and benchmark computational tasks in response to the forcing applied to the artificial muscles during actuation. The propulsor's nonlinear fluid-structural dynamics act as the physical reservoir and the kinematic feedback serves as the reservoir readouts. We show that the bio-inspired underwater propulsor can predict the hydrodynamic thrust and benchmark tasks with high accuracy under specific input frequencies. By analyzing the frequency spectrum of the input, readouts, and target signals, we demonstrate that the system's dynamic response determines the frequency contents relevant to the task being predicted. The propulsor's ability to process information stems from its nonlinearity, as it is responsible to transform the input signal into a broader spectrum of frequency content at the readouts. This broad band of frequency content is necessary to recreate the target signal within the PRC algorithm, thereby improving the prediction performance. The spectral analysis provides a unique perspective to analyze the nonlinear dynamics of a physical reservoir and serves as a valuable tool for examining other types of vibratory systems for PRC. This work serves as a first step towards embodying computation into soft bio-inspired swimmers.

Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

General reinforcement learning control for AUV manoeuvring in turbulent flows

Learning Agile Swimming: An End-to-End Approach without CPGs

Active particles using reinforcement learning to navigate in complex motility landscapes

Learning Efficient Navigation in Vortical Flow Fields

Learning to swim in potential flow

Smart inertial particles

Learning to swim efficiently in a nonuniform flow field

Steering undulatory micro-swimmers in a fluid flow through reinforcement learning

Physical reservoir computing on a soft bio-inspired swimmer

Fine Tuning Swimming Locomotion Learned from Mosquito Larvae

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Short term vs. long term: optimization of microswimmer navigation on different time horizons

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

Swimming Strategy of Settling Elongated Micro-Swimmers by Reinforcement Learning

A physics-informed reinforcement learning approach for the interfacial area transport in two-phase flow

Motion control of autonomous underwater vehicle based on physics-informed offline reinforcement learning

Autonomous underwater vehicle path planning based on actor-multi-critic reinforcement learning

Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control

A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary–lattice Boltzmann method