Abstract:Abstract Multi-agent multi-target search strategies can be utilized in complex scenarios such as post-disaster search and rescue by unmanned aerial vehicles. To solve the problem of fixed target and trajectory, the current multi-agent multi-target search strategies are mainly based on deep reinforcement learning (DRL). However, the training of agents by the DRL tend to be brittle due to their sensitivity to the training environment, which makes the strategies learned by the agents fall into local optima frequently, resulting in poor system robustness. Additionally, sparse rewards in DRL will lead to the problems such as difficulty in system convergence and low utilization efficiency of the sampled data. To address the problem that the robustness of the agents is weakened and the sparse rewards exist in the multi-objective search environment, we propose a MiniMax Multi-agent Deep Deterministic Policy Gradient based on the Parallel Hindsight Experience Replay (PHER-M3DDPG) algorithm, which adopts the framework of centralized training and decentralized execution in continuous action space. To enhance the system robustness, the PHER-M3DDPG algorithm employs a minimax learning architecture, which adaptively adjusts the learning strategy of agents by involving adversarial disturbances. In addition, to solve the sparse rewards problem, the PHER-M3DDPG algorithm adopts a parallel hindsight experience replay mechanism to increase the efficiency of data utilization by involving virtual learning targets and batch processing of the sampled data. Simulation results show that the PHER-M3DDPG algorithm outperforms the existing algorithms in terms of convergence speed and the task completion time in a multi-target search environment.

A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning

Image Stitching and Target Perception for Autonomous Underwater Vehicle-collected Side-Scan Sonar Images

A Multi-Stage Visual Positioning Method for Autonomous Underwater Docking

Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning

Coordinated Target Localization and Tracking with Node Scheduling for a Network of Autonomous Underwater Vehicles

Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

Multi-Agent Reinforcement Learning Based Secure Searching and Data Collection in AUV Swarms.

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Hierarchical Heterogeneous Multi-Agent Cross-Domain Search Method Based on Deep Reinforcement Learning

Multi-AUV underwater static target search method based on consensus-based bundle algorithm and improved glasius bio-inspired neural network

An AUV Target-Tracking Method Combining Imitation Learning and Deep Reinforcement Learning

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Dynamic Target Assignment by Unmanned Surface Vehicles Based on Reinforcement Learning

A Multi-Robot Coverage Path Planning Method for Maritime Search and Rescue Using Multiple AUVs

HA-MARL: Heuristic and APF Assisted Multi-Agent Reinforcement Learning for Wireless Data Sharing in AUV Swarms

Deep Reinforcement Learning Based Multi-UUV Cooperative Control for Target Capturing

Multi-AUV Assisted Seamless Underwater Target Tracking Relying on Deep Learning and Reinforcement Learning

Multi-AUV Cooperative Localization in Adaptive Sampling for Marine Environmental Monitoring

A Path Planning Approach for Multi-AUV Systems with Concurrent Stationary Node Access and Adaptive Sampling

Multi-UAV Escape Target Search: A Multi-Agent Reinforcement Learning Method

Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment