Abstract:Recent advancements in reinforcement learning have witnessed remarkable achievements by intelligent agents ranging from game-playing to industrial applications. Of particular interest is the area of multi-agent reinforcement learning (MARL), which holds significant potential for real-world scenarios. However, typical MARL methods are limited in their ability to handle tens of agents, leaving scenarios with up to hundreds or even thousands of agents almost unexplored. The scaling up of the number of agents presents two primary challenges: (1) agent-agent interactions are crucial in multi-agent systems while the number of interactions grows quadratically with the number of agents, resulting in substantial computational complexity and difficulty in strategies-learning; (2) the strengths of interactions among agents exhibit variations both across agents and over time, making it difficult to precisely model such interactions. In this paper, we propose a novel approach named Graph Attention Mean Field (GAT-MF). By converting agent-agent interactions into interactions between each agent and a weighted mean field, we achieve a substantial reduction in computational complexity. The proposed method offers a precise modeling of interaction dynamics with mathematical proofs of its correctness. Additionally, we design a graph attention mechanism to automatically capture the diverse and time-varying strengths of interactions, ensuring an accurate representation of agent interactions. Through extensive experimentation conducted in both manual and real-world scenarios involving over 3000 agents, we validate the efficacy of our method. The results demonstrate that our method outperforms the best baseline method with a remarkable improvement of 42.7%. Furthermore, our method saves 86.4% training time and 19.2% GPU memory compared to the best baseline method.

MassNE: Exploring Higher-Order Interactions with Marginal Effect for Massive Battle Outcome Prediction

NeuralAC: Learning Cooperation and Competition Effects for Match Outcome Prediction.

Estimation of Peer Influence Effect in Online Games Using Machine Learning Approaches.

Multi-Dimensional Prediction of Guild Health in Online Games: A Stability-Aware Multi-Task Learning Approach

Multi-source Data Multi-task Learning for Profiling Players in Online Games

A Generic Multi-Player Transformation Algorithm for Solving Large-Scale Zero-Sum Extensive-Form Adversarial Team Games

Globally Optimized Matchmaking in Online Games

Predicting Wargame Outcomes and Evaluating Player Performance from an Integrated Strategic and Operational Perspective

A Machine Learning Approach to Detect Strategic Behavior from Large-Population Observational Data Applied to Game Mode Prediction on a Team-Based Video Game

Multiplayer battle game-inspired optimizer for complex optimization problems

Modeling Game Avatar Synergy and Opposition through Embedding in Multiplayer Online Battle Arena Games

GAT-MF: Graph Attention Mean Field for Very Large Scale Multi-Agent Reinforcement Learning

The NeurIPS 2022 Neural MMO Challenge: A Massively Multiagent Competition with Specialization and Trade

Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play

Individualized Context-Aware Tensor Factorization for Online Games Predictions

Massively Multiagent Minigames for Training Generalist Agents

Player Behavior and Optimal Team Composition for Online Multiplayer Games

Decentralized optimal large scale multi-player pursuit-evasion strategies: A mean field game approach with reinforcement learning

Building Endgame Data Set to Improve Opponent Modeling Approach.

Enhanced Equilibria-Solving via Private Information Pre-Branch Structure in Adversarial Team Games