Abstract:The autonomous navigation and obstacle avoidance capabilities of autonomous underwater vehicles (AUVs) are essential for ensuring their safe navigation and long-term, efficient operation. However, the complexity of the marine environment poses significant challenges to safe and effective obstacle avoidance. To address this issue, this study proposes an AUV obstacle avoidance control algorithm based on offline reinforcement learning. This method adopts the Conservative Q-learning (CQL) algorithm, which is based on the Soft Actor-Critic (SAC) framework. It learns from obtained historical obstacle avoidance data and ultimately achieves a favorable obstacle avoidance control strategy. In this method, PID and SAC control algorithms are utilized to generate expert obstacle avoidance data to construct a diversified offline database. Additionally, based on the line-of-sight (LOS) guidance method and artificial potential field (APF) method, information regarding the distance and orientation of targets and obstacles is incorporated into the state space, and heading and obstacle avoidance reward terms are integrated into the reward function design. The algorithm successfully guides the AUV in autonomous navigation and dynamic obstacle avoidance in three-dimensional space. Furthermore, the algorithm exhibits a certain degree of anti-interference capability against uncertain disturbances and ocean currents, enhancing the safety and robustness of the AUV system. Simulation results fully demonstrate the feasibility and effectiveness of the intelligent obstacle avoidance method based on offline reinforcement learning. This study highlights the profound significance of offline reinforcement learning in enabling robust and reliable control systems for AUVs, paving the way for enhanced operational capabilities in challenging marine environments.

An Obstacle Avoiding Control Strategy for Autonomous Underwater Vehicle

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

Path Following Based on Waypoints and Real-Time Obstacle Avoidance Control of an Autonomous Underwater Vehicle

Adaptive barrier Lyapunov function-based obstacle avoidance control for an autonomous underwater vehicle with multiple static and moving obstacles

Multi-AUV Control Method Based on Inverse Optimal Control of Integrated Obstacle Avoidance Algorithm

Behavior-based Collision Free Path Following Control of an Autonomous Underwater Vehicle

An Obstacle Avoidance Strategy for AUV Based on State-Tracking Collision Detection and Improved Artificial Potential Field

AUV Collision Avoidance Strategy Based on Fuzzy Reinforcement Learning

A Control Method for Path Following of AUVs Considering Multiple Factors Under Ocean Currents

Adaptive Dual-Channel Event-Triggered Fuzzy Control for Autonomous Underwater Vehicles with Multiple Obstacles Environment

An AUV fuzzy obstacle avoidance method under event feedback supervision

Path Planning under Constraints and Path Following Control of Autonomous Underwater Vehicle with Dynamical Uncertainties and Wave Disturbances

An Obstacle Avoidance Algorithm for AUV Based on Obstacle's Detected Outline

Path Parameter Consensus-Based Formation and Obstacle Avoidance Control of Autonomous Underwater Vehicles

Modeling and Motion Control Strategy for Autonomous Underwater Vehicles

Underactuated AUV Path Following Control Based on Adaptive Backstepping Method.

Obstacle Avoidance for a Large-Scale High-Speed Underactuated AUV in Complex Environments

Robust Adaptive Path Following Control for Autonomous Underwater Vehicles with Virtual Guidance

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Line of Sight Guidance with Intelligent Obstacle Avoidance for Autonomous Underwater Vehicles

Integral Vector Field Control for Three-Dimensional Path Following of Autonomous Underwater Vehicle