Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Neural Network H-infinity Robust Adaptive Control for Autonomous Underwater Vehicle in 3-Dimensional Path Following

Three-dimensional Path Tracking Control for Autonomous Underwater Vehicle Based on Neural Network

Adaptive Neural Network Control System of Path Following for AUVs

Three-Dimensional Path Following of an Underactuated AUV Based on Neuro-Adaptive Command Filtered Backstepping Control

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Neural Network Based Robust Adaptive Tracking Control for the Automomous Underwater Vehicle

Robust Fuzzy 3D Path Following for Autonomous Underwater Vehicle Subject to Uncertainties

Spatial Path Following for Auvs Using Adaptive Neural Network Controllers

Adaptive Neural Network Control of AUVs with Control Input Nonlinearities Using Reinforcement Learning

3D Trajectory Tracking Control of an Underactuated AUV Based on Adaptive Neural Network Dynamic Surface

Neural Network Based Robust Adaptive Dynamic Surface Control for AUVs

Three-dimensional Path Tracking Control for an Autonomous Underwater Vehicle Based on L-two Disturbance Attenuation Method

Robust Adaptive Optimal Trajectory Tracking Control for Underactuated AUVs with Position and Velocity Constraints in Three‐dimensional Space

Command filter based adaptive neural trajectory tracking control of an underactuated underwater vehicle in three-dimensional space

Neural Network Based Robust Controller for Trajectory Tracking of Underwater Vehicles

Robust Adaptive Path Following Control for Autonomous Underwater Vehicles with Virtual Guidance

Adaptive neural network control system of bottom following for an underactuated AUV

FNN-based L2 Following Control of Underactuated Autonomous Underwater Vehicles

Adaptive Neural Network Dynamic Surface Trajectory Tracking Control for Underactuated Autonomous Underwater Vehicles

Three-Dimensional Path Following of an Underactuated AUV Based on Fuzzy Backstepping Sliding Mode Control

Command-Filtered Robust Adaptive NN Control With the Prescribed Performance for the 3-D Trajectory Tracking of Underactuated AUVs