Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Distributed Optimal Formation Tracking Control Based on Reinforcement Learning for Underactuated AUVs with Asymmetric Constraints

Cooperative Learning Formation Control of Multiple Autonomous Underwater Vehicles with Prescribed Performance Based on Position Estimation

Three-Dimensional Coordination Control for Multiple Autonomous Underwater Vehicles

Distributed Path-Following Formation Control of Multi-AUV Based on Graph Laplacian

Distributed Lyapunov-Based Model Predictive Control for AUV Formation Systems with Multiple Constraints

Bounded Neural Adaptive Formation Control of Multiple Underactuated AUVs under Uncertain Dynamics.

Distributed Neurodynamics-Based Backstepping Optimal Control for Robust Constrained Consensus of Underactuated Underwater Vehicles Fleet

Adaptive Formation Learning Control for Cooperative AUVs under Complete Uncertainty

Bearing-Based Robust Formation Tracking Control of Underactuated AUVs With Optimal Parameter Tuning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Filter-backstepping Based Neural Adaptive Formation Control of Leader-Following Multiple AUVs in Three Dimensional Space

Sliding mode based neural adaptive formation control of underactuated AUVs with leader-follower strategy

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

A Consensus-Based Model Predictive Control with Optimized Line-of-Sight Guidance for Formation Trajectory Tracking of Autonomous Underwater Vehicles

Distributed robust model predictive control-based formation-containment tracking control for autonomous underwater vehicles

Formation Control of Autonomous Underwater Vehicles Using an Improved Nonlinear Backstepping Method

Learning-based Robust Optimal Tracking Controller Design for Unmanned Underwater Vehicles with Full-State and Input Constraints

Event-triggered output-feedback adaptive tracking control of autonomous underwater vehicles using reinforcement learning

Model-free Finite-Time Formation Containment Control of Underactuated UUVs Subject to Input Delays and Unknown Interaction Information

Adaptive Formation Motion Planning and Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning

Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel Via Reinforcement Learning