Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

A Plume-Tracing Strategy Via Continuous State-Action Reinforcement Learning

Plume Tracing Via Model-Free Reinforcement Learning Method

Learning an End-To-End Policy for AUV Control Within Just Forty Minutes Using Parallel Simulation

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Depth Control of Model-Free AUVs Via Reinforcement Learning

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Path Planning of Unmanned Underwater Vehicles Based on Deep Reinforcement Learning Algorithm

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning

Attention-Based Meta-Reinforcement Learning for Tracking Control of AUV with Time-Varying Dynamics

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

Research and Design of an Autonomous Underwater Vehicle Path Planning Method Based on Deep Reinforcement Learning

Intelligent AUV Surfacing Control in Network Attack Scenario

Path Planning based on Deep Reinforcement Learning for Autonomous Underwater Vehicles under Ocean Current Disturbance

Comprehensive Ocean Information-Enabled AUV Motion Planning Based on Reinforcement Learning

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method

Unmanned Surface Vehicle Aided Maritime Data Collection Using Deep Reinforcement Learning

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning