Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Path following of QAUV using attitude-velocity coupling model

State Estimation of QAUV Based on Dynamic Model

Controllability Analysis of a Quadrotor-like Autonomous Underwater Vehicle

USV Application Scenario Expansion Based on Motion Control, Path Following and Velocity Planning

Path Following of Autonomous Surface Vehicles with Line-of-sight and Nonlinear Model Predictive Control

A New Path Following Method of AUV Using Line-of-sight with Sideway

QUUV: A Quadrotor-Like Unmanned Underwater Vehicle with Thrusts Configured As X Shape

Improved Line-of-sight Nonlinear Trajectory Tracking Control of Autonomous Underwater Vehicle Exposed to High Variable Speed Ocean Currents

Path Following Control with Sideslip Reduction for Underactuated Unmanned Surface Vehicles

A Quadrotor-Like Unmanned Underwater Vehicle

Obstacle Avoidance Strategy with Path-following for USV Based on Obstacle-guidance Model Using 2D-Lidar in Urban River Conditions

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

A Path Following Control Method for Underactuated Unmanned Surface Vehicles Based on Output Redefinition

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Path Following Based on Waypoints and Real-Time Obstacle Avoidance Control of an Autonomous Underwater Vehicle

Distributed Path-Following Formation Control of Multi-AUV Based on Graph Laplacian

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Adaptive Line-Of-Sight guidance law based on vector fields path following for underactuated unmanned surface vehicle

Unmanned Surface Vessel–Unmanned Aerial Vehicle Cooperative Path Following Based on a Predictive Line of Sight Guidance Law

Velocity and Trajectory Tracking Control Model for Underactuated UUVs Through Coupling of Direct CFD and PID Control Algorithm

From Aerial to underwater: Modeling and trajectory tracking control of the underactuated quadrotor hybrid Aerial underwater vehicle