Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

An end-to-end sensor scheduling method based on D3QN for underwater passive tracking in UWSNs

Adaptive Sensor Scheduling for Target Tracking in Underwater Wireless Sensor Networks

Coordinated Target Localization and Tracking with Node Scheduling for a Network of Autonomous Underwater Vehicles

A Multi-Node Cooperative Bearing-Only Target Passive Tracking Algorithm Via UWSNs

Distributed Target Tracking in UWSNs under Stochastic Node Communication Scheme

Energy-efficient Localization and Target Tracking Via Underwater Mobile Sensor Networks

An Efficient Depth-Adjustment Deployment Scheme for Underwater Wireless Sensor Networks

Real-Time Data Transmission Scheduling Algorithm for Wireless Sensor Networks Based on Deep Q-Learning

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

An Artificial Measurements-Based Adaptive Filter for Energy-Efficient Target Tracking Via Underwater Wireless Sensor Networks

Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

Tracking a Duty-Cycled Autonomous Underwater Vehicle by Underwater Wireless Sensor Networks

An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking

Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach

End-to-end multi-sensor fusion method based on deep reinforcement learning in UASNs

Energy-efficient and delay-guaranteed routing algorithm for software-defined wireless sensor networks: A cooperative deep reinforcement learning approach

Routing Protocol Design for Underwater Optical Wireless Sensor Networks: A Multiagent Reinforcement Learning Approach.

An AUV-assisted Data Collection Scheme for UWSNs Based on Reinforcement Learning under the Influence of Ocean Current

Joint Link Scheduling and Power Allocation in Imperfect and Energy-Constrained Underwater Wireless Sensor Networks

Adaptive Sensor Scheduling Algorithm for Target Tracking in Wireless Sensor Networks