Trajectory tracking control of vectored thruster autonomous underwater vehicles based on deep reinforcement learning

Tao Liu Jintao Zhao Yuli Hu Junhao Huang a School of Ocean Engineering and Technology,Sun Yat-sen University & Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai),Zhuhai,People's Republic of Chinab School of Marine Science and Technology,Northwestern Polytechnical University,Xi'an,People's Republic of China
DOI: https://doi.org/10.1080/17445302.2024.2391235
2024-08-22
Ships and Offshore Structures
Abstract:Vectored thruster autonomous underwater vehicles (AUVs) offer superior manoeuvrability compared to traditional fin and rudder systems, especially at low or zero velocities. However, controlling these AUVs becomes challenging in the presence of unpredictable interference forces and external water flow velocities. This paper introduces a novel three-dimensional trajectory tracking method for vectored thruster AUVs using reinforcement learning, enabling the system to learn optimal control policies through environmental interaction. A 5-degree of freedom (5-DOF) model is developed from the kinematics and dynamics equations of the underactuated AUV. An extended state observer (ESO) is used to estimate the differential expansion state based on observed variables. A reinforcement learning-based trajectory tracking strategy is then implemented. Simulation experiments demonstrate the method's effectiveness in precisely controlling AUVs under varying environmental conditions, achieving exceptional tracking accuracy even in the presence of random interference forces and external water flow velocities.
engineering, marine
What problem does this paper attempt to address?