Abstract:Recent advancements in reinforcement learning have witnessed remarkable achievements by intelligent agents ranging from game-playing to industrial applications. Of particular interest is the area of multi-agent reinforcement learning (MARL), which holds significant potential for real-world scenarios. However, typical MARL methods are limited in their ability to handle tens of agents, leaving scenarios with up to hundreds or even thousands of agents almost unexplored. The scaling up of the number of agents presents two primary challenges: (1) agent-agent interactions are crucial in multi-agent systems while the number of interactions grows quadratically with the number of agents, resulting in substantial computational complexity and difficulty in strategies-learning; (2) the strengths of interactions among agents exhibit variations both across agents and over time, making it difficult to precisely model such interactions. In this paper, we propose a novel approach named Graph Attention Mean Field (GAT-MF). By converting agent-agent interactions into interactions between each agent and a weighted mean field, we achieve a substantial reduction in computational complexity. The proposed method offers a precise modeling of interaction dynamics with mathematical proofs of its correctness. Additionally, we design a graph attention mechanism to automatically capture the diverse and time-varying strengths of interactions, ensuring an accurate representation of agent interactions. Through extensive experimentation conducted in both manual and real-world scenarios involving over 3000 agents, we validate the efficacy of our method. The results demonstrate that our method outperforms the best baseline method with a remarkable improvement of 42.7%. Furthermore, our method saves 86.4% training time and 19.2% GPU memory compared to the best baseline method.

Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning

Learning Intra-group Cooperation in Multi-agent Systems.

Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning

A Graph-Based Soft Actor Critic Approach in Multi-Agent Reinforcement Learning

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

GAMA: Graph Attention Multi-agent reinforcement learning algorithm for cooperation

Graph Soft Actor-Critic Reinforcement Learning for Large-Scale Distributed Multirobot Coordination

GCMA: an Adaptive Multi-Agent Reinforcement Learning Framework with Group Communication for Complex and Similar Tasks Coordination

Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning.

Distributed Cooperative Multi-Agent Reinforcement Learning with Directed Coordination Graph

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

GHGC: Goal-based Hierarchical Group Communication in Multi-Agent Reinforcement Learning.

Cooperative multi-agent game based on reinforcement learning

Self-Clustering Hierarchical Multi-Agent Reinforcement Learning with Extensible Cooperation Graph

GAT-MF: Graph Attention Mean Field for Very Large Scale Multi-Agent Reinforcement Learning

Very Large Scale Multi-Agent Reinforcement Learning with Graph Attention Mean Field

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

BGC: Multi-Agent Group Belief with Graph Clustering

Coordination Between Individual Agents in Multi-Agent Reinforcement Learning.

Hierarchical relationship modeling in multi-agent reinforcement learning for mixed cooperative–competitive environments

Abbreviated Weighted Graph in Multi-Agent Reinforcement Learning.