Abstract:In this paper, the reinforcement learning method for cooperative multi-agent systems (MAS) with incremental num-ber of agents is studied. The existing multi-agent reinforcement learning approaches deal with the MAS with a specific number of agents, and can learn well-performed policies. However, if there is an increasing number of agents, the previously learned in may not perform well in the current scenario. The new agents need to learn from scratch to find optimal policies with others, which may slow down the learning speed of the whole team. To solve that problem, in this paper, we propose a new algorithm to take full advantage of the historical knowledge which was learned before, and transfer it from the previous agents to the new agents. Since the previous agents have been trained well in the source environment, they are treated as teacher agents in the target environment. Correspondingly, the new agents are called student agents. To enable the student agents to learn from the teacher agents, we first modify the input nodes of the networks for teacher agents to adapt to the current envi-ronment. Then, the teacher agents take the observations of the student agents as input, and output the advised actions and va-lues as supervising information. Finally, the student agents com-bine the reward from the environment and the supervising in-formation from the teacher agents, and learn the optimal policies with modified loss functions. By taking full advantage of the knowledge of teacher agents, the search space for the student agents will be reduced significantly, which can accelerate the learning speed of the holistic system. The proposed algorithm is verified in some multi-agent simulation environments, and its ef-ficiency has been demonstrated by the experiment results.

Selective Policy Transfer in Multi-Agent Systems with Sparse Interactions

Multi-Agent Policy Transfer Via Task Relationship Modeling.

Learning in Multi-Agent Systems with Sparse Interactions by Knowledge Transfer and Game Abstraction

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

ATA-MAOPT: Multi-Agent Online Policy Transfer Using Attention Mechanism With Time Abstraction

Value Function Transfer for Deep Multi-Agent Reinforcement Learning Based on N-Step Returns.

A Versatile Agent for Fast Learning from Human Instructors

Off-Agent Trust Region Policy Optimization

Order Matters: Agent-by-agent Policy Optimization.

Safe Adaptive Policy Transfer Reinforcement Learning for Distributed Multiagent Control.

Accurate policy detection and efficient knowledge reuse against multi-strategic opponents

Stochastic Ensemble Policy Transfer

Enabling Inter-Agent Transfer for Multi-Agent Learning System by Incorporating Role Reversal

Efficient Exploration for Multi-Agent Reinforcement Learning Via Transferable Successor Features

Research on Isomorphic Task Transfer Algorithm Based on Knowledge Distillation in Multi-Agent Collaborative Systems

Preference-based experience sharing scheme for multi-agent reinforcement learning in multi-target environments

Efficient Policy Detecting and Reusing for Non-Stationarity in Markov Games.

Knowledge Transfer in Multi-Agent Reinforcement Learning with Incremental Number of Agents

SPD: Synergy Pattern Diversifying Oriented Unsupervised Multi-agent Reinforcement Learning.

Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning

IOB: Integrating Optimization Transfer and Behavior Transfer for Multi-Policy Reuse