Abstract:The fish-like robot is one of the typical underwater robots, which has the advantage of high maneuverability with low noise due to its bioinspired structure and biomimetic locomotion. However, it is challenging to efficiently design motion controllers for such robots to achieve satisfactory performance on specific control tasks in the real underwater environment, since the complex fluid-structure interaction exists during their swimming and exact dynamic models are absent. In this article, we propose a learning framework, incorporating a simulation system and a training methodology, to autonomously and fast train in simulation to create control policies that are capable of directly applying to a type of physical fish-like robots to perform motion control tasks. First, we construct a simulation system combining a data-driven environment and a computational fluid dynamics (CFD)-based environment, thus well balancing the simulation accuracy and the calculation speed. Second, we design a training methodology to train deep reinforcement learning (DRL)-based policies for the robot in our constructed simulation system to perform a specific control task. Then, we use two typical motion control tasks to verify our proposed framework. One is the path-following control task, which is a one-objective problem with dense rewards, while the other is the pose control task which is a two-objective problem with sparse rewards. For each task, the DRL-based control policy trained by our learning framework is directly deployed on the physical fish-like robot to perform the task in the real world. Experimental results show that the policies trained in simulation still work well in the real world, and perform even better in terms of control accuracy and stability compared with the traditional control methods, thus demonstrating the effectiveness of our learning framework.

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

Design and implementation of a biomimetic robotic fish with 3D locomotion

A Data-Driven Motion Control Approach for a Robotic Fish

Motion Control of a Multi-Joint Robotic Fish Based on Biomimetic Learning

Generation of robotic fish locomotion through biomimetic learning

Learning Agile Swimming: An End-to-End Approach without CPGs

Gliding Motion Optimization for a Biomimetic Gliding Robotic Fish

Path-following Control of Fish-like Robots: A Deep Reinforcement Learning Approach

Learning for Attitude Holding of a Robotic Fish: An End-to-End Approach With Sim-to-Real Transfer

Realization and Online Optimization for Gliding and Flapping Propulsion of a Manta Ray Robot

From Simulation to Reality: A Learning Framework for Fish-Like Robots to Perform Control Tasks

Fish Can Save Energy Via Proprioceptive Sensing.

Efficient Navigation of a Robotic Fish Swimming Across the Vortical Flow Field

Design of Deep Reinforcement Learning Controller Through Data-assisted Model for Robotic Fish Speed Tracking

Leveraging Imitation Learning on Pose Regulation Problem of a Robotic Fish

Design and Control of an Agile Robotic Fish With Integrative Biomimetic Mechanisms

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Investigating the Influence of Counterflow Regions on the Hydrodynamic Performance of Biomimetic Robotic Fish

Recent Progress in Modeling and Control of Bio-Inspired Fish Robots

Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control

Learning swimming via deep reinforcement learning