Abstract:With the development of the applications of unmanned surface vehicles (USVs), USV automation technologies are attracting increasing attention. In the industry, through the subtask division, it is generally believed that course-keeping is a critical basic sub-system in a series of complex automation systems and affects USV automation performance to a great extent. By course-keeping, we mean USV adjusts its angle to the desired angle and keeps it. In recent decades, course-keeping has been mainly achieved through classical first principles technologies, such as proportion–integral–differential (PID) controllers, leading to extremely laborious parameter tuning, especially in changeable wave environments. With the emergence and extensive application of data-driven technologies, deep reinforcement learning is conspicuous in sequential decision-making tasks, but it introduces a lack of explainability and physical meaning. To take full advantage of the data-driven and first principles paradigm and easily extend to the industry, in this paper, we propose an intelligent adaptive PID controller enhanced by proximal policy optimization (PPO) to achieve USV high-level automation. We then further verify its performance in path-following tasks compared with the PID controller. The results demonstrate that the proposed controller inherits the merits of explainability from PID and excellent sequential decision making from PPO and possesses excellent disturbance rejection performance when facing the disturbance of a changeable wave environment.

Speed and Heading Control of an Unmanned Surface Vehicle Using Deep Reinforcement Learning

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

Cooperative Control of Velocity and Heading for Unmanned Surface Vessel Based on Twin Delayed Deep Deterministic Policy Gradient with an Integral Compensator

Research on Motion Attitude Control of Under-actuated Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Deep reinforcement learning-based controller for dynamic positioning of an unmanned surface vehicle

Autonomous Surface Vehicle Control Method Using Deep Reinforcement Learning

Model-based Deep Reinforcement Learning for Data-Driven Motion Control of an Under-Actuated Unmanned Surface Vehicle: Path Following and Trajectory Tracking

Deep Reinforcement Learning-Based Path Planning of Underactuated Surface Vessels

An Improved Method towards Multi-UAV Autonomous Navigation Using Deep Reinforcement Learning

Data-Driven Performance-Prescribed Reinforcement Learning Control of an Unmanned Surface Vehicle

Intelligent Controller for Unmanned Surface Vehicles by Deep Reinforcement Learning

UAV Visual Flight Control Method Based on Deep Reinforcement Learning

Robust Motion Control for UAV in Dynamic Uncertain Environments Using Deep Reinforcement Learning.

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

Unmanned Surface Vessel Motion Intention Recognition Based on Deep Reinforcement Learning

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle

Fuzzy Categorical Deep Reinforcement Learning of a Defensive Game for an Unmanned Surface Vessel

Learning to Control of an Under-actuated Autonomous Surface Vehicle Based on Model-based Deep Reinforcement Learning