Abstract:This work proposes an innovative path-following control method, anchored in deep reinforcement learning (DRL), for unmanned underwater vehicles (UUVs). This approach is driven by several new designs, all of which aim to enhance learning efficiency and effectiveness and achieve high-performance UUV control. Specifically, a novel experience replay strategy is designed and integrated within the twin-delayed deep deterministic policy gradient algorithm (TD3). It distinguishes the significance of stored transitions by making a trade-off between rewards and temporal-difference (TD) errors, thus enabling the UUV agent to explore optimal control policies more efficiently. Another major challenge within this control problem arises from action oscillations associated with DRL policies. This issue leads to excessive system wear on actuators and makes real-time application difficult. To mitigate this challenge, a newly improved regularization method is proposed, which provides a moderate level of smoothness to the control policy. Furthermore, a dynamic reward function featuring adaptive constraints is designed to avoid unproductive exploration and expedite learning convergence speed further. Simulation results show that our method garners higher rewards in fewer training episodes compared with mainstream DRL-based control approaches (e.g., deep deterministic policy gradient (DDPG) and vanilla TD3) in UUV applications. Moreover, it can adapt to varying path configurations amid uncertainties and disturbances, all while ensuring high tracking accuracy. Simulation and experimental studies are conducted to verify the effectiveness.

A Path Following Controller for Deep-Sea Mining Vehicles Considering Slip Control and Random Resistance Based on Improved Deep Deterministic Policy Gradient

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

AI-based Dynamic Avoidance in Deep-Sea Mining

AUV path following controlled by modified Deep Deterministic Policy Gradient

Path Following Control On Moving. Robot For Deep Sea-Bed Mining

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

Path-Tracking Control Strategy of Unmanned Vehicle Based on DDPG Algorithm

An Integrated Dynamic Model and Optimized Fuzzy Controller for Path Tracking of Deep-Sea Mining Vehicle

Algorithms for dynamic control of a deep-sea mining vehicle based on deep reinforcement learning

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

Path Tracking Control of Autonomous Ground Vehicles Via Model Predictive Control and Deep Deterministic Policy Gradient Algorithm

An Improved DQN Algorithm for Automated Guided Vehicle Pathfinding Problem in Port Environment

The Wide-Area Coverage Path Planning Strategy for Deep-Sea Mining Vehicle Cluster Based on Deep Reinforcement Learning

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle

Autonomous Surface Vehicle Control Method Using Deep Reinforcement Learning

Path-Following and Obstacle Avoidance Control of Nonholonomic Wheeled Mobile Robot Based on Deep Reinforcement Learning

Adaptive Dynamic Model-Based Path Following Controller Design for an Unmanned Surface Vessel

Trace tracking algorithm of deep sea mining vehicle

Three-Dimensional Path-Following Control of an Autonomous Underwater Vehicle Based on Deep Reinforcement Learning