Abstract:This paper addresses a learning-based path following control scheme for a biomimetic underwater vehicle (BUV) driven by undulatory fins. A dynamic line-of-sight (DLOS) guidance system is designed, which uses a virtual ball with a dynamic radius to detect the reference path. This DLOS system guides our BUV in the path following control and extracts essential information for the Markov decision process (MDP) of the control task. A deep reinforcement learning (DRL) algorithm, sample-observed soft actor-critic (SOSAC) is proposed. The can train out control policy with greater cumulative reward and higher success rate by using two tricks: sample observation and sample diversification. Based on the DLOS system, the MDP of the control task, and a multilayer perceptron (MLP) trained by the SOSAC, our control scheme is established. Experiments show that our BUV can successfully achieve path following control in an indoor pool environment by using this control scheme. Note to Practitioners—The motivation of this paper is to design a practical end-to-end path following control scheme for the BUV driven by undulatory fins, and verify this scheme in a real-world environment. Unlike common autonomous underwater vehicles (AUVs) using axial propellers, the BUVs apply biomimetic propellers such as the undulatory fin. Multimodel wave patterns can be implemented by the undulatory fin, which generates nonlinear thrust and lateral force simultaneously. This propulsive feature makes the driving force on different directions of the BUV to be strong coupled, and it is complicated to convert the outputs of a common controller into waveform parameters of the undulatory fins to control the BUV. Therefore, in this paper, we proposed an end-to-end learning-based path following controller, which observes environmental information and directly generates waveform parameters to control our BUV. Experiments suggest that our control scheme is practical and valid.

An Offline Reinforcement Learning Approach for Path Following of an Unmanned Surface Vehicle

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Surface path tracking method of autonomous surface underwater vehicle based on deep reinforcement learning

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Sample-Observed Soft Actor-Critic Learning for Path Following of a Biomimetic Underwater Vehicle

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Path Planning of Unmanned Underwater Vehicles Based on Deep Reinforcement Learning Algorithm

Reinforcement Learning-Based Tracking Control of USVs in Varying Operational Conditions

An Algorithm of Complete Coverage Path Planning for Unmanned Surface Vehicle Based on Reinforcement Learning

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Data-Driven Performance-Prescribed Reinforcement Learning Control of an Unmanned Surface Vehicle

Adaptive Dynamic Model-Based Path Following Controller Design for an Unmanned Surface Vessel

A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Robust path following on rivers using bootstrapped reinforcement learning

Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learning