Abstract:Unmanned Surface Vehicle (USV) has a broad application prospect and autonomous path planning as its crucial technology has developed into a hot research direction in the field of USV research. This paper proposes an Improved Dueling Deep Double-Q Network Based on Prioritized Experience Replay (IPD3QN) to address the slow and unstable convergence of traditional Deep Q Network (DQN) algorithms in autonomous path planning of USV. Firstly, we use the deep double Q-Network to decouple the selection and calculation of the target Q value action to eliminate overestimation. The prioritized experience replay method is adopted to extract experience samples from the experience replay unit, increase the utilization rate of actual samples, and accelerate the training speed of the neural network. Then, the neural network is optimized by introducing a dueling network structure. Finally, the soft update method is used to improve the stability of the algorithm, and the dynamic ε-greedy method is used to find the optimal strategy. The experiments are first conducted in the Open AI Gym test platform to pre-validate the algorithm for two classical control problems: the Cart pole and Mountain Car problems. The impact of algorithm hyperparameters on the model performance is analyzed in detail. The algorithm is then validated in the Maze environment. The comparative analysis of simulation experiments shows that IPD3QN has a significant improvement in learning performance regarding convergence speed and convergence stability compared with DQN, D3QN, PD2QN, PDQN, PD3QN. Also, USV can plan the optimal path according to the actual navigation environment with the IPD3QN algorithm.

A Hyper-Heuristic with Deep Q-network for the Multi-Objective Unmanned Surface Vehicles Scheduling Problem

Exact and Deep Q-Network Assisted Swarm Intelligence Methods for Scheduling Multi-Objective Heterogeneous Unmanned Surface Vehicles

Multiple Task Assignment and Path Planning of a Multiple Unmanned Surface Vehicles System Based on Improved Self-Organizing Mapping and Improved Genetic Algorithm

A Hybrid Path Planning Algorithm for Unmanned Surface Vehicles in Complex Environment with Dynamic Obstacles.

Optimal Path Planning of Unmanned Surface Vehicle under Current Environment

Global path planning algorithm based on double DQN for multi-tasks amphibious unmanned surface vehicle

A Path Planning Approach for Multi-AUV Systems with Concurrent Stationary Node Access and Adaptive Sampling

Solving Heterogeneous USV Scheduling Problems by Problem-Specific Knowledge Based Meta-Heuristics with Q-Learning

A path planning approach for unmanned surface vehicles based on dynamic and fast Q-learning

Multiple targets traversing for unmanned surface vehicles by bundled genetic optimization and fast-marching Q-Learning

An Optimal Control-Based Path Planning Method for Unmanned Surface Vehicles in Complex Environments

Cooperative Coverage Path Planning for AUVs in Integrated Underwater Acoustic Communication and Detection Networks

A new SFN scheme for DTMB system based on UTC reference time

An Algorithm of Complete Coverage Path Planning for Unmanned Surface Vehicle Based on Reinforcement Learning

A multi-objective bi-level task planning strategy for UUV target visitation in ocean environment

An Improved Dueling Deep Double-Q Network Based on Prioritized Experience Replay for Path Planning of Unmanned Surface Vehicles

Path planning for Multi-USV target coverage in complex environments

Global path planning and waypoint following for heterogeneous unmanned surface vehicles assisting inland water monitoring

Cooperative Path Planning of Multiple Unmanned Surface Vehicles for Search and Coverage Task

Implantable cardioverter defibrillators after acute myocardial infarction

Unmanned Surface Vehicle Aided Maritime Data Collection Using Deep Reinforcement Learning