Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Three-Dimensional Path-Following Control of a Robotic Airship with Reinforcement Learning

Three Dimensional Path-Following Control Of An Under-Actuated Airship

Model-Free Control for Stratospheric Airship Based on Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Three-Dimensional Path-Following Control of an Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Three dimensional path-following control for an under-actuated airship

Airship Control Based on Q-Learning Algorithm and Neural Network

Three Dimensional Path-Following Backstepping Control for An Underactuated Stratospheric Airship

Robust Three-Dimensional Path-Following Control for an Under-Actuated Stratospheric Airship

Lateral path controller design for autonomous airship

Mode-Free Altitude Control for Airship Based on Q-Learning and Cmac Network

Direct-adaptive Fuzzy Path Following Control for an Autonomous Airship

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

Planar Path Following Nonlinear Controller Design for an Autonomous Airship

Path Planning and Following Control of a Quadrotor Helicopter in Three-Dimensional Space With Limited Information

Three Dimension Path Following Control of Unmanned Airship Based on H∞ Robust Adaptive Method

Virtual guidance-based finite-time path-following control of underactuated autonomous airship with error constraints and uncertainties

Autonomous Airship Path Following Control: Theory and Experiments

Spatial Path Following Control of an Autonomous Underactuated Airship

Reliable Intelligent Path Following Control for a Robotic Airship Against Sensor Faults