Abstract:Autonomous Underwater Vehicles (AUVs), as a member of the unmanned intelligent ocean vehicle group, can replace human beings to complete dangerous tasks in the ocean. It is of great significance to apply reinforcement learning (RL) to AUVs to realize intelligent control. This paper proposes an AUV obstacle avoidance framework based on event-triggered reinforcement learning. Firstly, an environment perception model is designed to judge the relative position relationship between the AUV and all unknown obstacles and known targets. Secondly, considering that the detection range of AUVs is limited, and the proposed method needs to deal with unknown static obstacles and unknown dynamic obstacles at the same time, two different event-triggered mechanisms are designed. Soft actor–critic (SAC) with a non-policy sampling method is used. Then, improved reinforcement learning and the event-triggered mechanism are combined in this paper. Finally, a simulation experiment of the obstacle avoidance task is carried out on the Gazebo simulation platform. Results show that the proposed method can obtain higher rewards and complete tasks successfully. At the same time, the trajectory and the distance between each obstacle confirm that the AUV can reach the target well while maintaining a safe distance from static and dynamic obstacles.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the obstacle avoidance problem for Autonomous Underwater Vehicles (AUVs) in environments with unknown static and dynamic obstacles. Specifically, the paper proposes an AUV obstacle avoidance framework based on Event-Triggered Reinforcement Learning (ET-RL) to achieve intelligent control. The main objectives of the paper include: 1. **Environmental Perception**: Design an environmental perception model to determine the relative position relationship between the AUV and all unknown obstacles and known targets. 2. **Obstacle Avoidance within Limited Detection Range**: Considering the limited detection range of the AUV and the need to handle both unknown static and dynamic obstacles, two different event-triggered mechanisms are designed. 3. **Reinforcement Learning Method**: Combine an improved reinforcement learning algorithm (such as Soft Actor-Critic, SAC) with the event-triggered mechanism to construct an AUV obstacle avoidance framework. 4. **Experimental Validation**: Conduct simulation experiments of obstacle avoidance tasks on the Gazebo simulation platform to verify the effectiveness of the proposed method. ### Main Contributions 1. **Environmental State Perception Model**: Designed an environmental state perception model to model the environmental state around the underwater vehicle, capable of perceiving environmental state features. 2. **Event-Triggered Mechanism**: Designed two different event-triggered mechanisms for designing state space and reward functions, improving the obstacle avoidance performance of the AUV within a limited detection range. 3. **Improved Reinforcement Learning Algorithm**: Combined the event-triggered mechanism with the SAC algorithm, using an off-policy sampling method, and designed an AUV obstacle avoidance framework based on the improved SAC algorithm. 4. **Experimental Validation**: Conducted three sets of experiments on the constructed simulation platform to verify the effectiveness of the proposed method. The experiments included single obstacle avoidance, multiple obstacle avoidance, and multiple dynamic obstacle avoidance. ### Key Technologies - **Environmental Perception Model**: Calculate relative distance and relative yaw angle through a rotation matrix to model the relative position relationship between the AUV and obstacles or target areas. - **Event-Triggered Mechanism**: Designed two event-triggered mechanisms to handle static and dynamic obstacles, respectively. - **Reinforcement Learning Algorithm**: Used the SAC algorithm combined with the event-triggered mechanism to construct the obstacle avoidance framework. - **Reward Function**: Designed a reward function that comprehensively considers target proximity, obstacle avoidance safety, and special events. ### Experimental Results - **Single Obstacle Avoidance Experiment**: Verified the feasibility of the proposed method in a single obstacle environment. - **Multiple Obstacle Avoidance Experiment**: Verified the effectiveness of the proposed method in a multiple obstacle environment. - **Multiple Dynamic Obstacle Avoidance Experiment**: Further verified the ability of the proposed method to handle dynamic obstacles in a random environment. Through these experiments, the paper demonstrates that the proposed event-triggered reinforcement learning-based AUV obstacle avoidance framework can effectively complete obstacle avoidance tasks while maintaining a safe distance from both static and dynamic obstacles.

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Research on obstacle avoidance of underactuated autonomous underwater vehicle based on offline reinforcement learning

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Real-time Obstacle Avoidance for AUV Based on Reinforcement Learning and Dynamic Window Approach

Reinforcement Learning Based Obstacle Avoidance for Autonomous Underwater Vehicle

Intelligent AUV Surfacing Control in Network Attack Scenario

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraints

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

AUV Collision Avoidance Strategy Based on Fuzzy Reinforcement Learning

A Path Planning Method Based on Deep Reinforcement Learning for AUV in Complex Marine Environment

Guidance and Control of Autonomous Surface Underwater Vehicles for Target Tracking in Ocean Environment by Deep Reinforcement Learning

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

An AUV fuzzy obstacle avoidance method under event feedback supervision