Abstract:Finding the optimal game strategy is a difficult problem in unmanned aerial vehicle (UAV) swarm confrontation. As an effective solution to the sequential decision-making problem, multi-agent reinforcement learning (MARL) provides a promising way to realize intelligent countermeasures. However, there are two challenges in applying MARL to large-scale UAV swarm confrontation: i) the curse of dimensionality caused by the excessive scale of UAV clusters and ii) the generalization problem caused by the dynamically changing UAV cluster size. To address these problems, we propose a novel MARL paradigm, called Weighted Mean Field Reinforcement Learning , where the pairwise communication between any UAV and its neighbors is modeled as that between a central UAV and the virtual UAV, which is abstracted from the weighted mean effect of neighboring UAVs. This approach reduces the multi-agent problem to a two-agent problem, which can reduce the input dimension of the agent and adapt to the changing cluster size. The communication content between UAVs includes actions and local observations. Actions can enhance the cooperation between UAVs and alleviate the non-stationarity of the environment, while local observations can expand the perception range of the central UAV so that it can obtain more useful information about the environment. The attention mechanism is leveraged to enable UAVs to select more valuable information flexibly, making our method more scalable than other algorithms. Combining this paradigm with double Q-learning and actor-critic algorithms, we propose weighted mean field Q-learning (WMFQ) and weighted mean field actor-critic (WMFAC) algorithms. Experiments on our constructed UAV swarm confrontation environment verify the effectiveness and scalability of our algorithms.

Poster Abstract: Emergency Networking Using UAVs: A Reinforcement Learning Approach with Large Language Model

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Dense Multi-Agent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

Maximizing UAV Coverage in Maritime Wireless Networks: A Multiagent Reinforcement Learning Approach

Graph Convolutional Multi-Agent Reinforcement Learning For Uav Coverage Control

Deployment of Unmanned Aerial Vehicles in Next-Generation Wireless Communication Network Using Multi-Agent Reinforcement Learning

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

GAN-powered heterogeneous multi-agent reinforcement learning for UAV-assisted task offloading

Mean Field Deep Reinforcement Learning for Fair and Efficient UAV Control

DNN Task Assignment in UAV Networks: A Generative AI Enhanced Multi-Agent Reinforcement Learning Approach

Graph-Based Multi-agent Reinforcement Learning for Large-Scale UAVs Swarm System Control

Multi-Agent DRL for Air-to-Ground Communication Planning in UAV-Enabled IoT Networks

UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning

UAV-assisted fair communications for multi-pair users: A multi-agent deep reinforcement learning method

Joint Coverage and Power Control in Highly Dynamic and Massive UAV Networks: An Aggregative Game-theoretic Learning Approach

Reinforcement Learning in Multiple-UAV Networks: Deployment and Movement Design

Multi-UAV Collaborative Surveillance Network Recovery Via Deep Reinforcement Learning

Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Weighted Mean Field Reinforcement Learning for Large-Scale UAV Swarm Confrontation

Model-aided Federated Reinforcement Learning for Multi-UAV Trajectory Planning in IoT Networks