Abstract:The autonomous navigation and obstacle avoidance capabilities of autonomous underwater vehicles (AUVs) are essential for ensuring their safe navigation and long-term, efficient operation. However, the complexity of the marine environment poses significant challenges to safe and effective obstacle avoidance. To address this issue, this study proposes an AUV obstacle avoidance control algorithm based on offline reinforcement learning. This method adopts the Conservative Q-learning (CQL) algorithm, which is based on the Soft Actor-Critic (SAC) framework. It learns from obtained historical obstacle avoidance data and ultimately achieves a favorable obstacle avoidance control strategy. In this method, PID and SAC control algorithms are utilized to generate expert obstacle avoidance data to construct a diversified offline database. Additionally, based on the line-of-sight (LOS) guidance method and artificial potential field (APF) method, information regarding the distance and orientation of targets and obstacles is incorporated into the state space, and heading and obstacle avoidance reward terms are integrated into the reward function design. The algorithm successfully guides the AUV in autonomous navigation and dynamic obstacle avoidance in three-dimensional space. Furthermore, the algorithm exhibits a certain degree of anti-interference capability against uncertain disturbances and ocean currents, enhancing the safety and robustness of the AUV system. Simulation results fully demonstrate the feasibility and effectiveness of the intelligent obstacle avoidance method based on offline reinforcement learning. This study highlights the profound significance of offline reinforcement learning in enabling robust and reliable control systems for AUVs, paving the way for enhanced operational capabilities in challenging marine environments.

Concise Deep Reinforcement Learning Obstacle Avoidance for Underactuated Unmanned Marine Vessels.

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

Reinforcement Learning Based Obstacle Avoidance for Autonomous Underwater Vehicle

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

A COLREGs-Compliant Deep Reinforcement Learning Approach

Optimizing Multi-Vessel Collision Avoidance Decision Making for Autonomous Surface Vessels: A COLREGs-Compliant Deep Reinforcement Learning Approach

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Research on Unmanned Surface Vehicle Collision Avoidance Based on Deep Reinforcement Learning

Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments

A Novel Reinforcement Learning Collision Avoidance Algorithm for USVs Based on Maneuvering Characteristics and COLREGs

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Real-time Obstacle Avoidance for AUV Based on Reinforcement Learning and Dynamic Window Approach

A Balanced Collision Avoidance Algorithm for USVs in Complex Environment: A Deep Reinforcement Learning Approach

Obstacle avoidance USV in multi-static obstacle environments based on a deep reinforcement learning approach

Fuzzy Categorical Deep Reinforcement Learning of a Defensive Game for an Unmanned Surface Vessel

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Method for Intelligent Obstacle Avoidance Decision-Making of Unmanned Vessel in Unknown Waters

Obstacle Avoidance Algorithm for Mobile Robot Based on Deep Reinforcement Learning in Dynamic Environments.