Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Deep Reinforcement Learning Based Energy Efficient Underwater Acoustic Communications

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Exploiting Propagation Delay in Underwater Acoustic Communication Networks via Deep Reinforcement Learning

Deep Reinforcement Learning for Artificial Upwelling Energy Management

Environment-aware communication channel quality prediction for underwater acoustic transmissions: A machine learning method

Multi-agent deep reinforcement learning based multiple access for underwater cognitive acoustic sensor networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

AUV-Aided Optical-Acoustic Hybrid Data Collection Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Optical and Acoustic Dual Channel Multiple Access in Heterogeneous Underwater Sensor Networks

Power Allocation Based on Multi-Agent Deep Deterministic Policy Gradient for Underwater Acoustic Communication Networks

A Hierarchical Deep-Reinforcement Approach

An end-to-end sensor scheduling method based on D3QN for underwater passive tracking in UWSNs

Deep Reinforcement Learning-aided Transmission Design for Energy-efficient Link Optimization in Vehicular Communications

ADRP-DQL: An adaptive distributed routing protocol for underwater acoustic sensor networks using deep Q-learning

Adaptive Switching for Multimodal Underwater Acoustic Communications Based on Reinforcement Learning

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

Deep Reinforcement Learning for Channel and Power Allocation in UAV-enabled IoT Systems

Deep Reinforcement Learning Aided Intelligent Access Control in Energy Harvesting Based WLAN

Deep Reinforcement Learning Optimal Transmission Algorithm for Cognitive Internet of Things with RF Energy Harvesting