Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation

Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning

Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation

Benchmarking Safe Exploration in Deep Reinforcement Learning

Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

Benchmarking Deep Reinforcement Learning for Navigation in Denied Sensor Environments

Vision-driven UAV River Following: Benchmarking with Safe Reinforcement Learning

A Deep Safe Reinforcement Learning Approach for Mapless Navigation.

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Deep Reinforcement Learning for Navigation in AAA Video Games

Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

Robust path following on rivers using bootstrapped reinforcement learning

Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

A Simulator and First Reinforcement Learning Results for Underwater Mapping

Monocular Camera and Single-Beam Sonar-Based Underwater Collision-Free Navigation with Domain Randomization

A Real-time Algorithm for USV Navigation Based on Deep Reinforcement Learning

Reinforcement Learning from Demonstrations by Novel Interactive Expert and Application to Automatic Berthing Control Systems for Unmanned Surface Vessel