Abstract:Autonomous underwater vehicles (AUVs) are widely used in sampling on-site the seawater parameters, such as temperature, salinity and biomass for better understanding the ocean. The AUV path needs to be carefully planned in order to maximize the sampled information within the power constraints, which is known as the informative path planning (IPP). The existence of ocean currents further complicates the problem. This article proposes an IPP method for AUVs under the influence of ocean currents via combining the probabilistic roadmap and $Q$ -learning. Specifically, the $Q$ -learning algorithm builds an informative optimal AUV path by traversing a learned and updated $Q$ -table. The $Q$ -value in the table represents the expectation of the obtained reward if taking a certain action moving from one position to another. Considering the characteristics of the IPP task, we design the reward matrix in $Q$ -learning using the prior knowledge on the environment information. A convergent $Q$ -table guarantees that only one complete training and learning is required to generate the path between any two positions. This feature facilitates converting the possible repetitive path plannings into simple search problems, and thus the automatic return is easily realized whenever the AUV residual energy is insufficient. Moreover, to improve the efficiency of the $Q$ -learning algorithm, a probabilistic roadmap with random sampling is generated and combined with the $Q$ -learning. Various simulations and comparisons are carried out. The results demonstrate the effectiveness of the proposed IPP algorithm, showing that the convergence of the path planning can be achieved quickly and successfully. The superiority in terms of efficient return path planning over the traditional path planning method, RRT*, is also demonstrated.

AUV path following controlled by modified Deep Deterministic Policy Gradient

Learning an End-To-End Policy for AUV Control Within Just Forty Minutes Using Parallel Simulation

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.

Learning and Sampling-Based Informative Path Planning for AUVs in Ocean Current Fields

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

Neural-network-based Deterministic Policy Gradient for Depth Control of AUVs

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Multi Pseudo Q-learning Based Deterministic Policy Gradient for Tracking Control of Autonomous Underwater Vehicles

AUV path tracking with real-time obstacle avoidance via reinforcement learning under adaptive constraints

A Multi-Source-Data-Assisted AUV for Path Cruising: An Energy-Efficient DDPG Approach

A Path Planning Approach for Multi-AUV Systems with Concurrent Stationary Node Access and Adaptive Sampling

Fixed-Time Path-Following-Based Underactuated Unmanned Surface Vehicle Dynamic Positioning Control

Distributed Path-Following Formation Control of Multi-AUV Based on Graph Laplacian

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

LSTM-DPPO based deep reinforcement learning controller for path following optimization of unmanned surface vehicle

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Path Following Based on Waypoints and Real-Time Obstacle Avoidance Control of an Autonomous Underwater Vehicle

Lane Following Method Based on Improved DDPG Algorithm

Path Following Method for AUV Based on Q-Learning and RBF Neural Network