Abstract:This paper addresses a learning-based path following control scheme for a biomimetic underwater vehicle (BUV) driven by undulatory fins. A dynamic line-of-sight (DLOS) guidance system is designed, which uses a virtual ball with a dynamic radius to detect the reference path. This DLOS system guides our BUV in the path following control and extracts essential information for the Markov decision process (MDP) of the control task. A deep reinforcement learning (DRL) algorithm, sample-observed soft actor-critic (SOSAC) is proposed. The can train out control policy with greater cumulative reward and higher success rate by using two tricks: sample observation and sample diversification. Based on the DLOS system, the MDP of the control task, and a multilayer perceptron (MLP) trained by the SOSAC, our control scheme is established. Experiments show that our BUV can successfully achieve path following control in an indoor pool environment by using this control scheme. Note to Practitioners—The motivation of this paper is to design a practical end-to-end path following control scheme for the BUV driven by undulatory fins, and verify this scheme in a real-world environment. Unlike common autonomous underwater vehicles (AUVs) using axial propellers, the BUVs apply biomimetic propellers such as the undulatory fin. Multimodel wave patterns can be implemented by the undulatory fin, which generates nonlinear thrust and lateral force simultaneously. This propulsive feature makes the driving force on different directions of the BUV to be strong coupled, and it is complicated to convert the outputs of a common controller into waveform parameters of the undulatory fins to control the BUV. Therefore, in this paper, we proposed an end-to-end learning-based path following controller, which observes environmental information and directly generates waveform parameters to control our BUV. Experiments suggest that our control scheme is practical and valid.

Path-following Control of Fish-like Robots: A Deep Reinforcement Learning Approach

A Data-Driven Motion Control Approach for a Robotic Fish

Motion Control of a Multi-Joint Robotic Fish Based on Biomimetic Learning

From Simulation to Reality: A Learning Framework for Fish-Like Robots to Perform Control Tasks

Design and implementation of a biomimetic robotic fish with 3D locomotion

Generation of robotic fish locomotion through biomimetic learning

Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning

Motion control of a robotic fish via learning control approach with self-adaption

Speed Trajectory Tracking Of A Robotic Fish Based On Iterative Learning Control Approach

Precise Speed Tracking Control of a Robotic Fish Via Iterative Learning Control

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

Learning for Attitude Holding of a Robotic Fish: An End-to-End Approach With Sim-to-Real Transfer

Learning Agile Swimming: An End-to-End Approach without CPGs

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

A numerical simulation of target-directed swimming for a three-link bionic fish with deep reinforcement learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Design of Deep Reinforcement Learning Controller Through Data-assisted Model for Robotic Fish Speed Tracking

Sample-Observed Soft Actor-Critic Learning for Path Following of a Biomimetic Underwater Vehicle

Leveraging Imitation Learning on Pose Regulation Problem of a Robotic Fish

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle