Abstract:Recent advancements in reinforcement learning have witnessed remarkable achievements by intelligent agents ranging from game-playing to industrial applications. Of particular interest is the area of multi-agent reinforcement learning (MARL), which holds significant potential for real-world scenarios. However, typical MARL methods are limited in their ability to handle tens of agents, leaving scenarios with up to hundreds or even thousands of agents almost unexplored. The scaling up of the number of agents presents two primary challenges: (1) agent-agent interactions are crucial in multi-agent systems while the number of interactions grows quadratically with the number of agents, resulting in substantial computational complexity and difficulty in strategies-learning; (2) the strengths of interactions among agents exhibit variations both across agents and over time, making it difficult to precisely model such interactions. In this paper, we propose a novel approach named Graph Attention Mean Field (GAT-MF). By converting agent-agent interactions into interactions between each agent and a weighted mean field, we achieve a substantial reduction in computational complexity. The proposed method offers a precise modeling of interaction dynamics with mathematical proofs of its correctness. Additionally, we design a graph attention mechanism to automatically capture the diverse and time-varying strengths of interactions, ensuring an accurate representation of agent interactions. Through extensive experimentation conducted in both manual and real-world scenarios involving over 3000 agents, we validate the efficacy of our method. The results demonstrate that our method outperforms the best baseline method with a remarkable improvement of 42.7%. Furthermore, our method saves 86.4% training time and 19.2% GPU memory compared to the best baseline method.

MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning

Deep Multi-Agent Reinforcement Learning with Relevance Graphs

Dueling Network Architecture for Multi-Agent Deep Deterministic Policy Gradient

A Graph-Based Soft Actor Critic Approach in Multi-Agent Reinforcement Learning

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Graph MADDPG with RNN for multiagent cooperative environment

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

GAMA: Graph Attention Multi-agent reinforcement learning algorithm for cooperation

Graph Neural Networks with Model-based Reinforcement Learning for Multi-agent Systems

Multi-agent Deep Reinforcement Learning Based on Maximum Entropy

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

Efficient Policy Generation in Multi-Agent Systems via Hypergraph Neural Network

Very Large Scale Multi-Agent Reinforcement Learning with Graph Attention Mean Field

Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attention

Multi-Agent Mean Field Predict Reinforcement Learning

GAT-MF: Graph Attention Mean Field for Very Large Scale Multi-Agent Reinforcement Learning

Multi-Agent Partial Observable Safe Reinforcement Learning for Counter Uncrewed Aerial Systems

Graph Neural Network Meets Multi-Agent Reinforcement Learning: Fundamentals, Applications, and Future Directions

Graph-QMIX: Addressing the Partial Observation Issues Via Graph Neural Network in Multi-Agent Reinforcement Learning

Multi-Agent Actor-Critic with Hierarchical Graph Attention Network

Learning Attentional Communication with a Common Network for Multiagent Reinforcement Learning.