Abstract:In a complex underwater environment, finding a viable, collision-free path for an autonomous underwater vehicle (AUV) is a challenging task. The purpose of this paper is to establish a safe, real-time, and robust method of collision avoidance that improves the autonomy of AUVs. We propose a method based on active sonar, which utilizes a deep reinforcement learning algorithm to learn the processed sonar information to navigate the AUV in an uncertain environment. We compare the performance of double deep Q-network algorithms with that of a genetic algorithm and deep learning. We propose a line-of-sight guidance method to mitigate abrupt changes in the yaw direction and smooth the heading changes when the AUV switches trajectory. The different experimental results show that the double deep Q-network algorithms ensure excellent collision avoidance performance. The effectiveness of the algorithm proposed in this paper was verified in three environments: random static, mixed static, and complex dynamic. The results show that the proposed algorithm has significant advantages over other algorithms in terms of success rate, collision avoidance performance, and generalization ability. The double deep Q-network algorithm proposed in this paper is superior to the genetic algorithm and deep learning in terms of the running time, total path, performance in avoiding collisions with moving obstacles, and planning time for each step. After the algorithm is trained in a simulated environment, it can still perform online learning according to the information of the environment after deployment and adjust the weight of the network in real-time. These results demonstrate that the proposed approach has significant potential for practical applications.

Heading Control for an Autonomous Underwater Vehicle Using ELM-based Q-learning

ELM and LOS Based Path Tracking for Autonomous Underwater Vehicles

Application Of Reinforcement Learning To Autonomous Heading Control For Bionic Underwater Robots

Research on Motion Attitude Control of Under-actuated Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Controller Design of an Autonomous Underwater Vehicle Using ELM-based Sliding Mode Control

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

High-Level Tracking of Autonomous Underwater Vehicles Based on Pseudo Averaged Q-Learning.

An Effective Motion. Control Based on 2-DOF PID and ELM for AUV

A Learning-Based Flexible Autonomous Motion Control Method for UAV in Dynamic Unknown Environments

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Depth Control of Autonomous Underwater Vehicle Using Deep Reinforcement Learning

Three-Dimensional Path-Following Control of an Autonomous Underwater Vehicle Based on Deep Reinforcement Learning

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

Sliding Mode Heading Control for AUV Based on Continuous Hybrid Model-Free and Model-Based Reinforcement Learning

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

Research on L1 Adaptive Control of Autonomous Underwater Vehicles with X-Rudder

Neural Network Based Reinforcement Learning Control of Autonomous Underwater Vehicles with Control Input Saturation

Target following for an autonomous underwater vehicle using regularized ELM-based reinforcement learning

Improved Q-Learning Control for Optimal Tracking of Underwater Vehicle Manipulator System