Abstract:Abstract Multi-agent multi-target search strategies can be utilized in complex scenarios such as post-disaster search and rescue by unmanned aerial vehicles. To solve the problem of fixed target and trajectory, the current multi-agent multi-target search strategies are mainly based on deep reinforcement learning (DRL). However, the training of agents by the DRL tend to be brittle due to their sensitivity to the training environment, which makes the strategies learned by the agents fall into local optima frequently, resulting in poor system robustness. Additionally, sparse rewards in DRL will lead to the problems such as difficulty in system convergence and low utilization efficiency of the sampled data. To address the problem that the robustness of the agents is weakened and the sparse rewards exist in the multi-objective search environment, we propose a MiniMax Multi-agent Deep Deterministic Policy Gradient based on the Parallel Hindsight Experience Replay (PHER-M3DDPG) algorithm, which adopts the framework of centralized training and decentralized execution in continuous action space. To enhance the system robustness, the PHER-M3DDPG algorithm employs a minimax learning architecture, which adaptively adjusts the learning strategy of agents by involving adversarial disturbances. In addition, to solve the sparse rewards problem, the PHER-M3DDPG algorithm adopts a parallel hindsight experience replay mechanism to increase the efficiency of data utilization by involving virtual learning targets and batch processing of the sampled data. Simulation results show that the PHER-M3DDPG algorithm outperforms the existing algorithms in terms of convergence speed and the task completion time in a multi-target search environment.

Adversarial Search and Tracking with Multiagent Reinforcement Learning in Sparsely Observable Environment

S2rl

S2RL: Do We Really Need to Perceive All States in Deep Multi-Agent Reinforcement Learning?

S2RL: DoWe Really Need to Perceive All States in Deep Multi-Agent Reinforcement Learning?

Formation Tracking of Spatiotemporal Multiagent Systems: A Decentralized Reinforcement Learning Approach

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Emergent Behaviors in Multi-Agent Target Acquisition

Cat-and-Mouse Satellite Dynamics: Divergent Adversarial Reinforcement Learning for Contested Multi-Agent Space Operations

Decentralized Multi-Agent Active Search and Tracking when Targets Outnumber Agents

Learning to Optimize State Estimation in Multi-agent Reinforcement Learning-based Collaborative Detection

Diffusion-Reinforcement Learning Hierarchical Motion Planning in Adversarial Multi-agent Games

Camouflage Adversarial Attacks on Multiple Agent Systems

Sparse Adversarial Attack in Multi-agent Reinforcement Learning

What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?

Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation

Boosting Exploration in Multi-Task Reinforcement Learning using Adversarial Networks

Towards Secure Multi-Agent Deep Reinforcement Learning: Adversarial Attacks and Countermeasures

Multi-UAV Pursuit-Evasion with Online Planning in Unknown Environments by Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for Wireless Networks Against Adversarial Communications

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach