Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Neural-network-based Reinforcement Learning Control for Path Following of Underactuated Ships

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Adaptive neural network asymptotic path-following control of underactuated ships with stochastic sea loads

Learning Control of Uncertain Ocean Surface Ship Dynamics Using Neural Networks

Deep Reinforcement Learning-Based Path Control and Optimization for Unmanned Ships

Robust Path Following Controller Design of Under-actuated Ships

Adaptive NN event-triggered control for path following of underactuated vessels with finite-time convergence

Adaptive Neural Network Iterative Sliding Mode Course Tracking Control for Unmanned Surface Vessels

Robust Neural Path-Following Control for Underactuated Ships with the DVS Obstacles Avoidance Guidance

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

Identification and Learning Control of Ocean Surface Ship Using Neural Networks

Adaptive Control of a Marine Vessel Based on Reinforcement Learning

Adaptive Neural Network Control For Course-Keeping Of Ships With Input Constraints

2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

A knowledge-free path planning approach for smart ships based on reinforcement learning

Event-Triggered Optimal Tracking Control for Underactuated Surface Vessels Via Neural Reinforcement Learning

Comparison of path following in ships using modern and traditional controllers

Adaptive Optimal Tracking Control for Uncertain Unmanned Surface Vessel Via Reinforcement Learning

Nonlinear Mmodel Predictive Controller with Disturbance Observer for Path Following of Underactuated Ships

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.