Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Autonomous Reinforcement Control of Visual Underwater Vehicles: Real-Time Experiments Using Computer Vision

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

End-To-End Sensorimotor Control Problems Of Auvs With Deep Reinforcement Learning

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

Hybrid offline-online reinforcement learning for obstacle avoidance in autonomous underwater vehicles

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

Target Search Control Of Auv In Underwater Environment With Deep Reinforcement Learning

Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller

Integrated Target Recognition and Grasp Control for the Underwater Vehicle–Manipulator System

A Fast Adaptive AUV Control Policy Based on Progressive Networks with Context Information

Reinforcement-Learning-Based Visual Servoing of Underwater Vehicle Dual-Manipulator System

Event-triggered output-feedback adaptive tracking control of autonomous underwater vehicles using reinforcement learning

Autonomous underwater vehicle link alignment control in unknown environments using reinforcement learning

Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

Collaborative task decision-making of multi-UUV in dynamic environments based on deep reinforcement learning

Control and Coordination of a SWARM of Unmanned Surface Vehicles using Deep Reinforcement Learning in ROS

Multi-AUV Assisted Seamless Underwater Target Tracking Relying on Deep Learning and Reinforcement Learning

Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning