Abstract:The autonomous navigation and obstacle avoidance capabilities of autonomous underwater vehicles (AUVs) are essential for ensuring their safe navigation and long-term, efficient operation. However, the complexity of the marine environment poses significant challenges to safe and effective obstacle avoidance. To address this issue, this study proposes an AUV obstacle avoidance control algorithm based on offline reinforcement learning. This method adopts the Conservative Q-learning (CQL) algorithm, which is based on the Soft Actor-Critic (SAC) framework. It learns from obtained historical obstacle avoidance data and ultimately achieves a favorable obstacle avoidance control strategy. In this method, PID and SAC control algorithms are utilized to generate expert obstacle avoidance data to construct a diversified offline database. Additionally, based on the line-of-sight (LOS) guidance method and artificial potential field (APF) method, information regarding the distance and orientation of targets and obstacles is incorporated into the state space, and heading and obstacle avoidance reward terms are integrated into the reward function design. The algorithm successfully guides the AUV in autonomous navigation and dynamic obstacle avoidance in three-dimensional space. Furthermore, the algorithm exhibits a certain degree of anti-interference capability against uncertain disturbances and ocean currents, enhancing the safety and robustness of the AUV system. Simulation results fully demonstrate the feasibility and effectiveness of the intelligent obstacle avoidance method based on offline reinforcement learning. This study highlights the profound significance of offline reinforcement learning in enabling robust and reliable control systems for AUVs, paving the way for enhanced operational capabilities in challenging marine environments.

EROAS: 3D Efficient Reactive Obstacle Avoidance System for Autonomous Underwater Vehicles using 2.5D Forward-Looking Sonar

First Results of A Novel and Low Power Forward Looking Sonar Technology for Small AUV's

First Results Of A Novel And Low Power Forward Looldng Sonar Technology For Small Auv'S

Research on Obstacle Detection and Avoidance of Autonomous Underwater Vehicle Based on Forward-Looking Sonar

Collision Detection and Avoidance for Underwater Vehicles Using Omnidirectional Vision

A Real-time Obstacle Avoidance Method for AUV Using a Multibeam Forward Looking Sonar

A 3D real-time obstacle avoidance method for an under-actuated AUV

Navigation in the Presence of Obstacles for an Agile Autonomous Underwater Vehicle

Underwater Localization and Mapping Based on Multi-Beam Forward Looking Sonar.

Sonar Imaging Simulation in Real-time Obstacle Avoidance Algorithm for AUV Based on Creator/Vega

Mapping and Obstacle Avoidance Using Multi-Range Sonar for BSA-AUV: Methodology and Lake Trial

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Research on Real Time Obstacle Avoidance Method for AUV Based on Combination of ViT-DPFN and MPC

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

Path Following, Obstacle Detection and Obstacle Avoidance for Thrusted Underwater Snake Robots

A Real-Time Reaction Obstacle Avoidance Algorithm for Autonomous Underwater Vehicles in Unknown Environments.

Intelligent obstacle avoidance algorithms for autonomous underwater vehicle

An Environment Information-Driven Online Bi-level Path Planning Algorithm for Underwater Search and Rescue AUV

Use of a low-cost forward-looking sonar for collision avoidance in small AUVs, analysis and experimental results

Path Following Based on Waypoints and Real-Time Obstacle Avoidance Control of an Autonomous Underwater Vehicle

Collaborative Aquatic Positioning System Utilising Multi-beam Sonar and Depth Sensors