Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Non-Linear Path-Following Control of Micro-AUV

A double-loop control framework for AUV trajectory tracking under model parameters uncertainties and time-varying currents

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

Improved Line-of-sight Nonlinear Trajectory Tracking Control of Autonomous Underwater Vehicle Exposed to High Variable Speed Ocean Currents

Path Following of Autonomous Surface Vehicles with Line-of-sight and Nonlinear Model Predictive Control

A New Path Following Method of AUV Using Line-of-sight with Sideway

Path Following Control with Sideslip Reduction for Underactuated Unmanned Surface Vehicles

A Path Following Control Method for Underactuated Unmanned Surface Vehicles Based on Output Redefinition

Path Following Based on Waypoints and Real-Time Obstacle Avoidance Control of an Autonomous Underwater Vehicle

Path following of an Underactuated AUV Based on Fuzzy Backstepping Sliding Mode Control

Distributed Path-Following Formation Control of Multi-AUV Based on Graph Laplacian

Trajectory Tracking Controller Design for Underactuated AUVs Based on Adaptive Robust Control

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A Control Method for Path Following of AUVs Considering Multiple Factors Under Ocean Currents

Path Following Model Predictive Control of a Coupled Autonomous Underwater Vehicle

Formation Path Following Control of Underactuated AUVs -- With Proofs

Three‐dimensional path following control of underactuated autonomous underwater vehicles with nonzero roll dynamics: A novel line‐of‐sight‐guided approach

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Adaptive barrier Lyapunov function-based obstacle avoidance control for an autonomous underwater vehicle with multiple static and moving obstacles

Path Following Method for AUV Based on Q-Learning and RBF Neural Network

Trajectory Linearization-Based Adaptive PLOS Path Following Control for Unmanned Surface Vehicle with Unknown Dynamics and Rudder Saturation