Abstract:As an emerging direction of multi-agent collaborative control technology, multiple autonomous underwater vehicle (multi-AUV) cooperative area search technology has played an important role in civilian fields such as marine resource exploration and development, marine rescue, and marine scientific expeditions, as well as in military fields such as mine countermeasures and military underwater reconnaissance. At present, as we continue to explore the ocean, the environment in which AUVs perform search tasks is mostly unknown, with many uncertainties such as obstacles, which places high demands on the autonomous decision-making capabilities of AUVs. Moreover, considering the limited detection capability of a single AUV in underwater environments, while the area searched by the AUV is constantly expanding, a single AUV cannot obtain global state information in real time and can only make behavioral decisions based on local observation information, which adversely affects the coordination between AUVs and the search efficiency of multi-AUV systems. Therefore, in order to face increasingly challenging search tasks, we adopt multi-agent reinforcement learning (MARL) to study the problem of multi-AUV cooperative area search from the perspective of improving autonomous decision-making capabilities and collaboration between AUVs. First, we modeled the search task as a decentralized partial observation Markov decision process (Dec-POMDP) and established a search information map. Each AUV updates the information map based on sonar detection information and information fusion between AUVs, and makes real-time decisions based on this to better address the problem of insufficient observation information caused by the weak perception ability of AUVs in underwater environments. Secondly, we established a multi-AUV cooperative area search system (MACASS), which employs a search strategy based on multi-agent reinforcement learning. The system combines various AUVs into a unified entity using a distributed control approach. During the execution of search tasks, each AUV can make action decisions based on sonar detection information and information exchange among AUVs in the system, utilizing the MARL-based search strategy. As a result, AUVs possess enhanced autonomy in decision-making, enabling them to better handle challenges such as limited detection capabilities and insufficient observational information.

Underwater Target Tracking Based on Interrupted Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Time-Saving MARL Approach

Underwater Target Tracking Based on Hierarchical Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Advantage-Attention Actor-Critic Approach

A Software-Defined MARL-Based Architecture for AUV Cluster Network to Enable Cooperative and Smart Underwater Target Tracking

Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning

Coordinated Target Localization and Tracking with Node Scheduling for a Network of Autonomous Underwater Vehicles

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

HA-MARL: Heuristic and APF Assisted Multi-Agent Reinforcement Learning for Wireless Data Sharing in AUV Swarms

Enhancing Underwater IoT Security: A Collaborative Pursuit Strategy Using Multi-Agent Reinforcement Learning

Multi-AUV Collaborative Data Collection and Trajectory Planning in Integrated Sensing and Communication for Underwater Acoustic Networks

Smart Underwater Pollution Detection Based on Graph-Based Multi-Agent Reinforcement Learning Towards AUV-Based Network ITS

High-Sample-Efficient Multiagent Reinforcement Learning for Navigation and Collision Avoidance of UAV Swarms in Multitask Environments

Deep Reinforcement Learning-based Multi-AUV Task Allocation Algorithm in Underwater Wireless Sensor Networks

Underwater Equipotential Line Tracking Based on Self-Attention Embedded Multiagent Reinforcement Learning Toward AUV-Based ITS

MARL-Based AUV Formation for Underwater Intelligent Autonomous Transport Systems Supported by 6G Network

Multi-AUVs Cooperative Target Search Based on Autonomous Cooperative Search Learning Algorithm

A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning

Correction Redistribution Mechanism Based on Forward-Reverse Solutions and Real-Time Path Dynamic Adaptive Re-Planning for Multi-AUVs Collaborative Search

Multi-AUV Collaborative Data Collection Algorithm Based on Q-Learning in Underwater Acoustic Sensor Networks

Multi-Agent Reinforcement Learning Based Secure Searching and Data Collection in AUV Swarms.

A Method for Multi-AUV Cooperative Area Search in Unknown Environment Based on Reinforcement Learning

Deep Reinforcement Learning Based Multi-UUV Cooperative Control for Target Capturing