Abstract:Autonomous underwater vehicles (AUVs) are widely used in sampling on-site the seawater parameters, such as temperature, salinity and biomass for better understanding the ocean. The AUV path needs to be carefully planned in order to maximize the sampled information within the power constraints, which is known as the informative path planning (IPP). The existence of ocean currents further complicates the problem. This article proposes an IPP method for AUVs under the influence of ocean currents via combining the probabilistic roadmap and $Q$ -learning. Specifically, the $Q$ -learning algorithm builds an informative optimal AUV path by traversing a learned and updated $Q$ -table. The $Q$ -value in the table represents the expectation of the obtained reward if taking a certain action moving from one position to another. Considering the characteristics of the IPP task, we design the reward matrix in $Q$ -learning using the prior knowledge on the environment information. A convergent $Q$ -table guarantees that only one complete training and learning is required to generate the path between any two positions. This feature facilitates converting the possible repetitive path plannings into simple search problems, and thus the automatic return is easily realized whenever the AUV residual energy is insufficient. Moreover, to improve the efficiency of the $Q$ -learning algorithm, a probabilistic roadmap with random sampling is generated and combined with the $Q$ -learning. Various simulations and comparisons are carried out. The results demonstrate the effectiveness of the proposed IPP algorithm, showing that the convergence of the path planning can be achieved quickly and successfully. The superiority in terms of efficient return path planning over the traditional path planning method, RRT*, is also demonstrated.

Binocular Vision-Based Motion Planning of An AUV: A Deep Reinforcement Learning Approach

Communication-Aware Motion Planning of AUV in Obstacle-Dense Environment: A Binocular Vision-Based Deep Learning Method.

Learning and Sampling-Based Informative Path Planning for AUVs in Ocean Current Fields

End-to-End AUV Motion Planning Method Based on Soft Actor-Critic

Comprehensive Ocean Information-Enabled AUV Motion Planning Based on Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

AUV Path Planning with Kinematic Constraints in Unknown Environment Using Reinforcement Learning.

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

An Information-Assisted Deep Reinforcement Learning Path Planning Scheme for Dynamic and Unknown Underwater Environment

A Motion Camouflage-Inspired Path Planning Method for UAVs Based on Reinforcement Learning

A Path Planning Approach for Multi-AUV Systems with Concurrent Stationary Node Access and Adaptive Sampling

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Adaptive Energy-Efficient Reinforcement Learning for AUV 3D Motion Planning in Complex Underwater Environments

Deep Learning-Based Nonparametric Identification and Path Planning for Autonomous Underwater Vehicles

Asynchronous Multithreading Reinforcement-Learning-Based Path Planning and Tracking for Unmanned Underwater Vehicle

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

An Intelligent Navigation Control Approach for Autonomous Unmanned Vehicles via Deep Learning-Enhanced Visual SLAM Framework

Path Planning based on Deep Reinforcement Learning for Autonomous Underwater Vehicles under Ocean Current Disturbance

Adaptive Formation Motion Planning and Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning