Abstract:Finding the optimal game strategy is a difficult problem in unmanned aerial vehicle (UAV) swarm confrontation. As an effective solution to the sequential decision-making problem, multi-agent reinforcement learning (MARL) provides a promising way to realize intelligent countermeasures. However, there are two challenges in applying MARL to large-scale UAV swarm confrontation: i) the curse of dimensionality caused by the excessive scale of UAV clusters and ii) the generalization problem caused by the dynamically changing UAV cluster size. To address these problems, we propose a novel MARL paradigm, called Weighted Mean Field Reinforcement Learning , where the pairwise communication between any UAV and its neighbors is modeled as that between a central UAV and the virtual UAV, which is abstracted from the weighted mean effect of neighboring UAVs. This approach reduces the multi-agent problem to a two-agent problem, which can reduce the input dimension of the agent and adapt to the changing cluster size. The communication content between UAVs includes actions and local observations. Actions can enhance the cooperation between UAVs and alleviate the non-stationarity of the environment, while local observations can expand the perception range of the central UAV so that it can obtain more useful information about the environment. The attention mechanism is leveraged to enable UAVs to select more valuable information flexibly, making our method more scalable than other algorithms. Combining this paradigm with double Q-learning and actor-critic algorithms, we propose weighted mean field Q-learning (WMFQ) and weighted mean field actor-critic (WMFAC) algorithms. Experiments on our constructed UAV swarm confrontation environment verify the effectiveness and scalability of our algorithms.

Multi-agent reinforcement learning aided intelligent UAV swarm for target tracking

Multi-Agent Reinforcement Learning Aided Intelligent UAV Swarm for Target Tracking

Multi-UAV Cooperative Target Tracking Based on Swarm Intelligence

Multi-target tracking for unmanned aerial vehicle swarms using deep reinforcement learning

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

UAV Swarm Cooperative Target Search: A Multi-Agent Reinforcement Learning Approach

Joint Communication and Action Learning in Multi-Target Tracking of UAV Swarms with Deep Reinforcement Learning

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Target Detection Formulti-Uavs Via Digital Pheromones and Navigation Algorithm in Unknownenvironments

Secure and Cooperative Target Tracking Via AUV Swarm - A Reinforcement Learning Approach.

Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

Deep Reinforcement Learning Based Multi-UUV Cooperative Control for Target Capturing

Multi-Target Pursuit by a Decentralized Heterogeneous UAV Swarm using Deep Multi-Agent Reinforcement Learning

Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning

Multi-UAV Collaborative Detection Based on Reinforcement Learning.

Graph-Based Multi-agent Reinforcement Learning for Large-Scale UAVs Swarm System Control

Cooperatively Pursuing a Target Unmanned Aerial Vehicle by Multiple Unmanned Aerial Vehicles Based on Multiagent Reinforcement Learning

UAV Multi-Dynamic Target Interception: A Hybrid Intelligent Method Using Deep Reinforcement Learning and Fuzzy Logic

Weighted Mean Field Reinforcement Learning for Large-Scale UAV Swarm Confrontation

Distributed Information Fusion Based Trajectory Tracking for USV and UAV Clusters Via Multi-Agent Deep Learning Approach

Collaborative strategy research of target tracking based on natural intelligence by UAV swarm