Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Neural Network Based Reinforcement Learning Control of Autonomous Underwater Vehicles with Control Input Saturation

Adaptive Neural Network Control of AUVs with Control Input Nonlinearities Using Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Deep Reinforcement Learning Based Optimal Trajectory Tracking Control of Autonomous Underwater Vehicle

Adaptive Neural Network Control of Underactuated Surface Vessels with Guaranteed Transient Performance: Theory and Experimental Results.

Neural Network Identification and Learning Control of Uncertain AUV

Adaptive Neural Network Iterative Sliding Mode Course Tracking Control for Unmanned Surface Vessels

Adaptive Neural Network Dynamic Surface Trajectory Tracking Control for Underactuated Autonomous Underwater Vehicles

Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel Via Reinforcement Learning

Neural Network Based Robust Adaptive Dynamic Surface Control for AUVs

Neural Network Based Robust Adaptive Tracking Control for the Automomous Underwater Vehicle

3D Trajectory Tracking Control of an Underactuated AUV Based on Adaptive Neural Network Dynamic Surface

Event-triggered output-feedback adaptive tracking control of autonomous underwater vehicles using reinforcement learning

Three-dimensional Path Tracking Control for Autonomous Underwater Vehicle Based on Neural Network

Adaptive neural network based Fault-tolerant trajectory tracking control of unmanned surface vessels with input saturation and error constraints

Data‐driven iterative learning cooperative trajectory tracking control for multiple autonomous underwater vehicles with input saturation constraints

Adaptive Neural Networks Command Filtered Control for AUVs with Input Saturation via Backstepping

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Adaptive neural network control system of bottom following for an underactuated AUV

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Adaptive Neural Network Control System of Path Following for AUVs