Abstract:This paper focuses on the dilemma faced by largescale multi-agent systems. With the increase of agent size, the policy search space grows exponentially, reaching the task complexity that is difficult for the current mainstream multiagent algorithms to deal with, so that the optimal strategy is converged in large-scale multi-agent systems. In order to make the learning process of large-scale multi-agent tasks more smooth, many researchers have introduced the method of course learning, setting the scale from small to large courses, compared with learning training from scratch, this way can continue to inherit the knowledge learned in the smaller scale tasks, making the training smoother. However, in order to realize the multiagent course learning from small to large scale, we face the following difficulties. First, the neural network structure we designed can adapt to the observation input of agents of different scales, rather than only applicable to a fixed scale. Second, in the multi-stage course learning, due to the network's inefficient representation of multi-agent systems, the knowledge learned in small-scale courses cannot be effectively utilized in large-scale courses, and even the problem of catastrophic forgetting occurs in the neural network. Specifically, we view multi-agent systems from a graph perspective and derive the Graph Decomposition Invariant (GDI) for the first time. We prove that neural network architectures satisfying the GDI are of great help in realizing efficient course learning. We design a model architecture called GDI model suitable for multi-agent curriculum learning, which satisfies the graph decomposition invariant proposed by us. Finally, we designed small-scale to large-scale multi-agent tasks on two experimental platforms, StarCraft and Neural MMO, and conducted a large number of comparative experiments, which verified that our proposed GDI model has a great promotion effect on course learning.

Graph based skill acquisition and transfer Learning for continuous reinforcement learning domains

GSC: A Graph-Based Skill Composition Framework for Robot Learning

Independent Skill Transfer for Deep Reinforcement Learning

Unsupervised Discovery of Transitional Skills for Deep Reinforcement Learning

Skill based transfer learning with domain adaptation for continuous reinforcement learning domains

DGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data

Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement

Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functions

Skill matters: Dynamic skill learning for multi-agent cooperative reinforcement learning

Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models

Graph learning-based generation of abstractions for reinforcement learning

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Towards Generalizability of Multi-Agent Reinforcement Learning in Graphs with Recurrent Message Passing

Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning

Learning Continually on a Sequence of Graphs -- The Dynamical System Way

E-CGL: An Efficient Continual Graph Learner

Learning Heuristics over Large Graphs via Deep Reinforcement Learning

Transferring Knowledge across Learning Processes

Learning state correspondence of reinforcement learning tasks for knowledge transfer

Scaling Up Multi-Agent Reinforcement Learning Via Graph Decomposition Invariant Network

Transfer in Deep Reinforcement Learning using Knowledge Graphs