Abstract:Autonomous underwater vehicles (AUVs) are widely used in sampling on-site the seawater parameters, such as temperature, salinity and biomass for better understanding the ocean. The AUV path needs to be carefully planned in order to maximize the sampled information within the power constraints, which is known as the informative path planning (IPP). The existence of ocean currents further complicates the problem. This article proposes an IPP method for AUVs under the influence of ocean currents via combining the probabilistic roadmap and $Q$ -learning. Specifically, the $Q$ -learning algorithm builds an informative optimal AUV path by traversing a learned and updated $Q$ -table. The $Q$ -value in the table represents the expectation of the obtained reward if taking a certain action moving from one position to another. Considering the characteristics of the IPP task, we design the reward matrix in $Q$ -learning using the prior knowledge on the environment information. A convergent $Q$ -table guarantees that only one complete training and learning is required to generate the path between any two positions. This feature facilitates converting the possible repetitive path plannings into simple search problems, and thus the automatic return is easily realized whenever the AUV residual energy is insufficient. Moreover, to improve the efficiency of the $Q$ -learning algorithm, a probabilistic roadmap with random sampling is generated and combined with the $Q$ -learning. Various simulations and comparisons are carried out. The results demonstrate the effectiveness of the proposed IPP algorithm, showing that the convergence of the path planning can be achieved quickly and successfully. The superiority in terms of efficient return path planning over the traditional path planning method, RRT*, is also demonstrated.

A Modified ALOS Method of Path Tracking for AUVs with Reinforcement Learning Accelerated by Dynamic Data-Driven AUV Model

A double-loop control framework for AUV trajectory tracking under model parameters uncertainties and time-varying currents

Improved Line-of-sight Nonlinear Trajectory Tracking Control of Autonomous Underwater Vehicle Exposed to High Variable Speed Ocean Currents

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Learning and Sampling-Based Informative Path Planning for AUVs in Ocean Current Fields

A New Path Following Method of AUV Using Line-of-sight with Sideway

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraints

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

A Multi-Source-Data-Assisted AUV for Path Cruising: An Energy-Efficient DDPG Approach

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV with Time-Varying Dynamics

Path Planning based on Deep Reinforcement Learning for Autonomous Underwater Vehicles under Ocean Current Disturbance

Improved Artificial Potential Field Algorithm Assisted by Multisource Data for AUV Path Planning

AUV Obstacle Avoidance Framework Based on Event-Triggered Reinforcement Learning

Asynchronous Multithreading Reinforcement-Learning-Based Path Planning and Tracking for Unmanned Underwater Vehicle

Learning-based Tracking Control of AUV: Mixed Policy Improvement and Game-Based Disturbance Rejection

Path Following Method for AUV Based on Q-Learning and RBF Neural Network

Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method with Experimental Validation.