Abstract:Autonomous underwater vehicles (AUVs) have become important tools in the ocean exploration and have drawn considerable attention. Precise control for AUVs is the prerequisite to effectively execute underwater tasks. However, the classical control methods such as model predictive control (MPC) rely heavily on the dynamics model of the controlled system which is difficult to obtain for AUVs. To address this issue, a new reinforcement learning (RL) framework for AUV path-following control is proposed in this paper. Specifically, we propose a novel actor-model-critic (AMC) architecture integrating a neural network model with the traditional actor-critic architecture. The neural network model is designed to learn the state transition function to explore the spatio-temporal change patterns of the AUV as well as the surrounding environment. Based on the AMC architecture, a RL-based controller agent named ModelPPO is constructed to control the AUV. With the required sailing speed achieved by a traditional proportional-integral (PI) controller, ModelPPO can control the rudder and elevator fins so that the AUV follows the desired path. Finally, a simulation platform is built to evaluate the performance of the proposed method that is compared with MPC and other RL-based methods. The obtained results demonstrate that the proposed method can achieve better performance than other methods, which demonstrate the great potential of the advanced artificial intelligence methods in solving the traditional motion control problems for intelligent vehicles.

Mpc-Based Model-Based Reinforcement Learning and Planning for Auv Path Following

Deep Reinforcement Learning with Model Predictive Control for Path Following of Autonomous Underwater Vehicle

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A Model Predictive Control Approach for USV Autonomous Cruising Via Disturbance Learning

Model Predictive Control for an Auv with Dynamic Path Planning

Autonomous Underwater Vehicle (AUV) Motion Design: Integrated Path Planning and Trajectory Tracking Based on Model Predictive Control (MPC)

A Path Planning Method Based on Deep Reinforcement Learning for AUV in Complex Marine Environment

A Model Predictive Path Following Control Method for Underactuated Autonomous Underwater Vehicles

Model Predictive Path Following Control of Underwater Vehicle Based on RBF Neural Network

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Path planning and obstacle avoidance control of UUV based on an enhanced A* algorithm and MPC in dynamic environment

Comprehensive Ocean Information-Enabled AUV Motion Planning Based on Reinforcement Learning

Computationally Efficient MPC for Path Following of Underactuated Marine Vessels Using Projection Neural Network

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

Action Guidance-Based Deep Interactive Reinforcement Learning for AUV Path Planning

Parameter Identification and Model Prediction Path Following Control of Underactuated AUV: Methodology and Experimental Verification

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Path-Following Control of an Auv Using Multi-Objective Model Predictive Control

Experimental Evaluation on Depth Control Using Improved Model Predictive Control for Autonomous Underwater Vehicle (AUVs).

Comparison of Linear and Nonlinear Model Predictive Control in Path Following of Underactuated Unmanned Surface Vehicles

Path Following Based on Waypoints and Real-Time Obstacle Avoidance Control of an Autonomous Underwater Vehicle