Abstract:The autonomous navigation and obstacle avoidance capabilities of autonomous underwater vehicles (AUVs) are essential for ensuring their safe navigation and long-term, efficient operation. However, the complexity of the marine environment poses significant challenges to safe and effective obstacle avoidance. To address this issue, this study proposes an AUV obstacle avoidance control algorithm based on offline reinforcement learning. This method adopts the Conservative Q-learning (CQL) algorithm, which is based on the Soft Actor-Critic (SAC) framework. It learns from obtained historical obstacle avoidance data and ultimately achieves a favorable obstacle avoidance control strategy. In this method, PID and SAC control algorithms are utilized to generate expert obstacle avoidance data to construct a diversified offline database. Additionally, based on the line-of-sight (LOS) guidance method and artificial potential field (APF) method, information regarding the distance and orientation of targets and obstacles is incorporated into the state space, and heading and obstacle avoidance reward terms are integrated into the reward function design. The algorithm successfully guides the AUV in autonomous navigation and dynamic obstacle avoidance in three-dimensional space. Furthermore, the algorithm exhibits a certain degree of anti-interference capability against uncertain disturbances and ocean currents, enhancing the safety and robustness of the AUV system. Simulation results fully demonstrate the feasibility and effectiveness of the intelligent obstacle avoidance method based on offline reinforcement learning. This study highlights the profound significance of offline reinforcement learning in enabling robust and reliable control systems for AUVs, paving the way for enhanced operational capabilities in challenging marine environments.

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A hybrid RVO-MPPI approach for efficient collision avoidance for multiple autonomous underwater vehicles

Motion control of autonomous underwater vehicle based on physics-informed offline reinforcement learning

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

Adaptive Formation Learning Control for Cooperative AUVs under Complete Uncertainty

Multi-AUV Pursuit-Evasion Game in the Internet of Underwater Things: an Efficient Training Framework Via Offline Reinforcement Learning

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Collision Avoidance and Path Point Tracking Control for Underactuated Unmanned Surface Vehicles with Unknown Model Nonlinearity

AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraints

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Adaptive Formation Motion Planning and Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

Reinforcement Learning Based Obstacle Avoidance for Autonomous Underwater Vehicle

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Multi-Objective-Optimization Multi-AUV Assisted Data Collection Framework for IoUT Based on Offline Reinforcement Learning

Adaptive barrier Lyapunov function-based obstacle avoidance control for an autonomous underwater vehicle with multiple static and moving obstacles