Abstract:The localization of underwater acoustic sensor networks (UASNs) has emerged as a critical research area in the marine information fusion field. Generally, the convex optimization method is adopted to solve the localization problem. However, this method has limitations in complex underwater environments, since it is difficult to transform the non-convex optimization problem into a convex optimization problem under such conditions. Recently, deep reinforcement learning (DRL) has shown great potential and promise in solving intricate optimization tasks. Motivated by this, we propose to adopt DRL for UASNs localization to improve accuracy and robustness. The key challenge is that existing DRL-based methods require discretization of the environment, which leads to a compromise between search time and localization precision. To address this challenge, we first model the localization problem as a Markov Decision Process (MDP) with continuous state and action spaces and subsequently introduce a continuous control DRL framework to solve the localization problem. Within this framework, we develop three continuous control DRL-based localization estimators to address the localization problem in unsupervised, supervised, and semisupervised scenarios. Comprehensive simulations demonstrate the effectiveness of our approach, as the proposed solutions exhibit several advantageous features compared to traditional methods, such as: 1) compared with convex optimization-based method, the convex relaxation is not required; 2) compared with least squares method, the proposed estimators are capable of converging to a global optimal state; 3) compared with discrete control DRL method, the proposed estimators reduce localization time and enhance localization accuracy significantly.

Joint Localisation and Tracking for Autonomous Underwater Vehicle: a Reinforcement Learning-Based Approach

A double-loop control framework for AUV trajectory tracking under model parameters uncertainties and time-varying currents

Integrated Localization and Tracking for AUV With Model Uncertainties via Scalable Sampling-Based Reinforcement Learning Approach

Coordinated Target Localization and Tracking with Node Scheduling for a Network of Autonomous Underwater Vehicles

Joint Localization and Tracking Design for AUV with Asynchronous Clocks and State Disturbances

Improved Line-of-sight Nonlinear Trajectory Tracking Control of Autonomous Underwater Vehicle Exposed to High Variable Speed Ocean Currents

An Underactuated AUV Tracking Algorithm Based on Backstepping Adaptive Sliding Mode Control

Cooperative Learning Formation Control of Multiple Autonomous Underwater Vehicles with Prescribed Performance Based on Position Estimation

An Improved Localization Method for the Transition Between Autonomous Underwater Vehicle Homing and Docking.

AUV-Aided Localization for Internet of Underwater Things: A Reinforcement-Learning-Based Method.

Multi-AUV Cooperative Localization in Adaptive Sampling for Marine Environmental Monitoring

Slide Mode-Based Joint Localization and Tracking of a Single AUV

Tracking Control of Autonomous Underwater Vehicles with Acoustic Localization and Extended Kalman Filter

Asynchronous Localization for Underwater Acoustic Sensor Networks: A Continuous Control Deep Reinforcement Learning Approach

AUV-Aided Localization for Underwater Acoustic Sensor Networks with Current Field Estimation

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Cyber-Physical Cooperative Localization Algorithms for Underwater Vehicle with Asynchronous Time Clock

Fuzzy-based Tracking Controller Design for Autonomous Underwater Vehicle

A Method of State Estimation for Underwater Vehicle Navigation Around a Cylindrical Structure

Multi-AUV Assisted Seamless Underwater Target Tracking Relying on Deep Learning and Reinforcement Learning

Joint Design of Channel Estimation and Flocking Control for Multi-AUV-Based Maritime Transportation Systems