Abstract:In the field of ocean energy detection, Autonomous Underwater Vehicles (AUVs) offer significant advantages in terms of manpower, resource, and energy efficiency. However, the unpredictable nature of the ocean environment, particularly the real-time changes in ocean currents, poses navigational risks for AUVs. Therefore, effective path planning in dynamic environments is crucial for AUVs to perform specific tasks. This paper addresses the static path planning problem and proposes a model called the noise net double DQN network with prioritized experience replay (N-DDQNP). The N-DDQNP model combines a noise network and a prioritized experience replay mechanism to address the limited exploration and slow convergence speed issues of the DQN algorithm, which are caused by the greedy strategy and uniform sampling mechanism. The proposed approach involves constructing a double DQN network with a priority experience replay and an exploration mechanism using the noise network. Second, a compound reward function is formulated to take into account ocean current, distance, and safety factors, ensuring prompt feedback during the training process. Regarding the ocean current, the reward function is designed based on the angle between the current direction and the AUV's heading direction, considering its impact on the AUV's speed. As for the distance factor, the reward is determined by the Euclidean distance between the current position and the target point. Furthermore, the safety factor considers whether the AUV may collide with obstacles. By incorporating these three factors, the compound reward function is established. To evaluate the performance of the N-DDQNP model, experiments were conducted using real ocean data in various complex ocean environments. The results demonstrate that the path planning time of the N-DDQNP model outperforms other algorithms in different ocean current scenarios and obstacle environments. Furthermore, a user console-AUV connection has been established using spice cloud desktop technology. The cloud desktop architecture enables intuitive observation of the AUV's navigation posture and the surrounding marine environment, facilitating safer and more efficient underwater exploration and marine resource detection tasks.

A Navigation Method for UUVs under Ocean Current Disturbance Based on Deep Reinforcement Learning

Learning and Sampling-Based Informative Path Planning for AUVs in Ocean Current Fields

UUV Navigation and Positioning Method Based on Model-Aided and Ocean Current Estimation

Path Planning based on Deep Reinforcement Learning for Autonomous Underwater Vehicles under Ocean Current Disturbance

UUV dynamic model-based navigation method

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Research on Geomagnetic Perceiving Navigation Method Based on Deep Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

A Path Planning Method Based on Deep Reinforcement Learning for AUV in Complex Marine Environment

An AUV-assisted Data Collection Scheme for UWSNs Based on Reinforcement Learning under the Influence of Ocean Current

Deep Reinforcement Learning Based Accurate UUV Localization

An Information-Assisted Deep Reinforcement Learning Path Planning Scheme for Dynamic and Unknown Underwater Environment

A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances.

End-to-end Navigation for Autonomous Underwater Vehicle with Hybrid Recurrent Neural Networks

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Continuous Control for Autonomous Underwater Vehicle Path Following Using Deep Interactive Reinforcement Learning

Path Planning of Unmanned Underwater Vehicles Based on Deep Reinforcement Learning Algorithm

Dynamic Data Collection of AUV Based on Deep Reinforcement Learning

AUV Obstacle Avoidance Planning Based on Deep Reinforcement Learning

An AUV-Assisted Data Gathering Scheme Based on Deep Reinforcement Learning for IoUT

AUV Path Planning Considering Ocean Current Disturbance Based on Cloud Desktop Technology