Cooperative Control of Velocity and Heading for Unmanned Surface Vessel Based on Twin Delayed Deep Deterministic Policy Gradient with an Integral Compensator

Yibai Wang,Shulong Zhao,Qingling Wang
DOI: https://doi.org/10.1016/j.oceaneng.2023.115943
IF: 5
2023-01-01
Ocean Engineering
Abstract:This paper addresses cooperative control of velocity and heading for an unmanned surface vessel (USV) utilizing a twin delay deep deterministic policy gradient (TD3) reinforcement learning algorithm. The utilization of a deep neural network establishes a direct correlation between the USV’s state parameters and motor control quantities. A reward function is devised to update the network parameters and which acquires the trained model. The introducing of an integral compensator effectively eliminates the steady-state error of the system, thereby significantly enhancing the precision of both velocity control and heading control. Furthermore, a two-stage training algorithm comprising offline learning and online learning has been devised. Through offline learning, a deep neural network model for the USV controller is obtained. Subsequently, the optimization of the controller strategy is conducted during the online learning phase. Ultimately, the simulation results demonstrate the exceptional control performance attained by the proposed algorithm.
What problem does this paper attempt to address?