Abstract:This paper focuses on the dilemma faced by largescale multi-agent systems. With the increase of agent size, the policy search space grows exponentially, reaching the task complexity that is difficult for the current mainstream multiagent algorithms to deal with, so that the optimal strategy is converged in large-scale multi-agent systems. In order to make the learning process of large-scale multi-agent tasks more smooth, many researchers have introduced the method of course learning, setting the scale from small to large courses, compared with learning training from scratch, this way can continue to inherit the knowledge learned in the smaller scale tasks, making the training smoother. However, in order to realize the multiagent course learning from small to large scale, we face the following difficulties. First, the neural network structure we designed can adapt to the observation input of agents of different scales, rather than only applicable to a fixed scale. Second, in the multi-stage course learning, due to the network's inefficient representation of multi-agent systems, the knowledge learned in small-scale courses cannot be effectively utilized in large-scale courses, and even the problem of catastrophic forgetting occurs in the neural network. Specifically, we view multi-agent systems from a graph perspective and derive the Graph Decomposition Invariant (GDI) for the first time. We prove that neural network architectures satisfying the GDI are of great help in realizing efficient course learning. We design a model architecture called GDI model suitable for multi-agent curriculum learning, which satisfies the graph decomposition invariant proposed by us. Finally, we designed small-scale to large-scale multi-agent tasks on two experimental platforms, StarCraft and Neural MMO, and conducted a large number of comparative experiments, which verified that our proposed GDI model has a great promotion effect on course learning.

Team Formation in Large Organizations: A Deep Reinforcement Learning Approach

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Negotiating Team Formation Using Deep Reinforcement Learning

Hierarchical Collaborative Resource Scheduling in Industrial Internet of Things Based on Graph Neural Networks and Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Graph Enhanced Reinforcement Learning for Effective Group Formation in Collaborative Problem Solving

Learning to Dynamically Coordinate Multi-Robot Teams in Graph Attention Networks

Large-scale Machine Learning Cluster Scheduling via Multi-agent Graph Reinforcement Learning

Learning to Transfer Role Assignment Across Team Sizes

Scaling Up Multi-Agent Reinforcement Learning Via Graph Decomposition Invariant Network

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning.

A Reinforcement Learning-based DAG Tasks Scheduling in Edge-Cloud Collaboration Systems

Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning

Reorganizing Complex Network to Improve Large-Scale Multiagent Teamwork

A Reinforcement Learning-assisted Genetic Programming Algorithm for Team Formation Problem Considering Person-Job Matching

A new multi-agent reinforcement learning approach

Combining graph neural network with deep reinforcement learning for resource allocation in computing force networks

An End-to-End Deep RL Framework for Task Arrangement in Crowdsourcing Platforms

Scalable Multi-Robot Task Allocation Using Graph Deep Reinforcement Learning with Graph Normalization

C3F: Constant Collaboration and Communication Framework for Graph-Representation Dynamic Multi-Robotic Systems