Abstract:In a complex underwater environment, finding a viable, collision-free path for an autonomous underwater vehicle (AUV) is a challenging task. The purpose of this paper is to establish a safe, real-time, and robust method of collision avoidance that improves the autonomy of AUVs. We propose a method based on active sonar, which utilizes a deep reinforcement learning algorithm to learn the processed sonar information to navigate the AUV in an uncertain environment. We compare the performance of double deep Q-network algorithms with that of a genetic algorithm and deep learning. We propose a line-of-sight guidance method to mitigate abrupt changes in the yaw direction and smooth the heading changes when the AUV switches trajectory. The different experimental results show that the double deep Q-network algorithms ensure excellent collision avoidance performance. The effectiveness of the algorithm proposed in this paper was verified in three environments: random static, mixed static, and complex dynamic. The results show that the proposed algorithm has significant advantages over other algorithms in terms of success rate, collision avoidance performance, and generalization ability. The double deep Q-network algorithm proposed in this paper is superior to the genetic algorithm and deep learning in terms of the running time, total path, performance in avoiding collisions with moving obstacles, and planning time for each step. After the algorithm is trained in a simulated environment, it can still perform online learning according to the information of the environment after deployment and adjust the weight of the network in real-time. These results demonstrate that the proposed approach has significant potential for practical applications.

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

Unmanned Surface Vehicle Aided Maritime Data Collection Using Deep Reinforcement Learning

Using Deep Reinforcement Learning Methods for Autonomous Vessels in 2D Environments

Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Real-time Planning and Collision Avoidance Control Method Based on Deep Reinforcement Learning

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

Dynamic Navigation and Area Assignment of Multiple USVs Based on Multi-Agent Deep Reinforcement Learning

A Balanced Collision Avoidance Algorithm for USVs in Complex Environment: A Deep Reinforcement Learning Approach

Deep reinforcement learning-based controller for dynamic positioning of an unmanned surface vehicle

Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

A Real-time Algorithm for USV Navigation Based on Deep Reinforcement Learning

Model-based Deep Reinforcement Learning for Data-Driven Motion Control of an Under-Actuated Unmanned Surface Vehicle: Path Following and Trajectory Tracking

Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

Path Planning for Underactuated Unmanned Surface Vehicle Swarm Based on Deep Reinforcement Learning

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Double Broad Reinforcement Learning Based on Hindsight Experience Replay for Collision Avoidance of Unmanned Surface Vehicles