Abstract:Recent advancements in reinforcement learning have witnessed remarkable achievements by intelligent agents ranging from game-playing to industrial applications. Of particular interest is the area of multi-agent reinforcement learning (MARL), which holds significant potential for real-world scenarios. However, typical MARL methods are limited in their ability to handle tens of agents, leaving scenarios with up to hundreds or even thousands of agents almost unexplored. The scaling up of the number of agents presents two primary challenges: (1) agent-agent interactions are crucial in multi-agent systems while the number of interactions grows quadratically with the number of agents, resulting in substantial computational complexity and difficulty in strategies-learning; (2) the strengths of interactions among agents exhibit variations both across agents and over time, making it difficult to precisely model such interactions. In this paper, we propose a novel approach named Graph Attention Mean Field (GAT-MF). By converting agent-agent interactions into interactions between each agent and a weighted mean field, we achieve a substantial reduction in computational complexity. The proposed method offers a precise modeling of interaction dynamics with mathematical proofs of its correctness. Additionally, we design a graph attention mechanism to automatically capture the diverse and time-varying strengths of interactions, ensuring an accurate representation of agent interactions. Through extensive experimentation conducted in both manual and real-world scenarios involving over 3000 agents, we validate the efficacy of our method. The results demonstrate that our method outperforms the best baseline method with a remarkable improvement of 42.7%. Furthermore, our method saves 86.4% training time and 19.2% GPU memory compared to the best baseline method.

Multi-agent air combat with two-stage graph-attention communication

Graph MADDPG with RNN for multiagent cooperative environment

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.

Learning Effective Communication for Cooperative Pursuit with Multi-Agent Reinforcement Learning

Game of Drones: Intelligent Online Decision Making of Multi-UAV Confrontation

Learning Multi-Agent Communication from Graph Modeling Perspective

Multi-UAV Cooperative Air Combat Decision-Making Based on Multi-Agent Double-Soft Actor-Critic

Learning Attentional Communication for Multi-Agent Cooperation

Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

Communication Learning in Multi-Agent Systems from Graph Modeling Perspective

Graph Attention-based Reinforcement Learning for Trajectory Design and Resource Assignment in Multi-UAV Assisted Communication

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Attention Enhanced Reinforcement Learning for Multi agent Cooperation

Cascaded Attention: Adaptive and Gated Graph Attention Network for Multiagent Reinforcement Learning

Multi-agent Cooperative Combat Simulation in Naval Battlefield with Reinforcement Learning

Multi-Agent Combat in Non-Stationary Environments

Graphcomm: a graph neural network based method for multi-agent reinforcement learning

Evolutionary reinforcement learning algorithm for large-scale multi-agent cooperation and confrontation applications

GAT-MF: Graph Attention Mean Field for Very Large Scale Multi-Agent Reinforcement Learning