Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Impact of tight optical filtering on orthogonal time-frequency domain multiplexed signals in wavelength-selective switching systems

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

Robust path following on rivers using bootstrapped reinforcement learning

AI on the Water: Applying DRL to Autonomous Vessel Navigation

Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment

2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

Survey of Deep Learning for Autonomous Surface Vehicles in Marine Environments

Using Deep Reinforcement Learning Methods for Autonomous Vessels in 2D Environments

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

Deep reinforcement learning for adaptive path planning and control of an autonomous underwater vehicle

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

Surface path tracking method of autonomous surface underwater vehicle based on deep reinforcement learning

Aeolus Ocean -- A simulation environment for the autonomous COLREG-compliant navigation of Unmanned Surface Vehicles using Deep Reinforcement Learning and Maritime Object Detection

Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learning

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning

Path planning of autonomous underwater vehicle in unknown environment based on improved deep reinforcement learning

Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles

Obstacle avoidance for environmentally-driven USVs based on deep reinforcement learning in large-scale uncertain environments