Abstract:Autonomous underwater vehicles (AUVs) play a crucial role in data collection for underwater acoustic sensor networks (UWASNs). The limited capacity of individual AUV and the need for low-latency data collection necessitate the deployment of AUV swarms to achieve efficient and secure cooperative data collection. However, most existing works assume prior knowledge of sensor node locations, which is impractical in real-world AUV networks. Additionally, continuous data collection needs to be considered due to the sustained operation of sensors and cluster head replacement. To address these challenges, we propose a target uncertainty map assisted data collection scheme for AUV swarms based on the multi-agent proximal policy optimization (MAPPO) algorithm. Specifically, the target uncertainty map is established by leveraging current and past search and collection results, guiding the AUV swarm to prioritize areas with higher probabilities of containing sensor nodes. Moreover, a digital pheromone mechanism incorporating repulsive and attractive pheromones is designed to establish an artificial potential field for adjusting the target uncertainty map. To further enable a comprehensive exploration of unknown environments, we introduce the age of information (AoI) as an indicator. Additionally, we consider the energy consumption associated with data collection to strike a balance between collection and energy efficiency, and derive a lower bound on the policy improvement achieved by the MAPPO algorithm. Simulation results have validated that the proposed scheme has a superior performance compared to the baselines, achieving an approximately 15% increase in the collection rate while reducing the energy consumption of data collection and AoI as well.

HA-MARL: Heuristic and APF Assisted Multi-Agent Reinforcement Learning for Wireless Data Sharing in AUV Swarms

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning Based Secure Searching and Data Collection in AUV Swarms.

Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following

MARL-based Design of Multi-Unmanned Aerial Vehicle Assisted Communication System with Hybrid Gaming Mode

Multiple Ships Cooperative Navigation and Collision Avoidance using Multi-agent Reinforcement Learning with Communication

Underwater Searching and Multi-Round Data Collection Via AUV Swarms: an Energy-Efficient AoI-Aware MAPPO Approach

Maximizing UAV Coverage in Maritime Wireless Networks: A Multiagent Reinforcement Learning Approach

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Multi-Agent Reinforcement Learning-Based Resource Sharing in Multi-UAV Wireless Networks

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

Multi-Agent Deep Reinforcement Learning Framework Strategized by Unmanned Aerial Vehicles for Multi-Vessel Full Communication Connection

Multi-AUV Pursuit-Evasion Game in the Internet of Underwater Things: an Efficient Training Framework Via Offline Reinforcement Learning

Multi-Objective-Optimization Multi-AUV Assisted Data Collection Framework for IoUT Based on Offline Reinforcement Learning

A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning

Dense Multi-Agent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

An Improved PPO for Multiple Unmanned Aerial Vehicles

Enhancing Information Freshness: An AoI Optimized Markov Decision Process Dedicated In the Underwater Task