Abstract:Multi-agent reinforcement learning shines as the pinnacle of multi-agent systems, conquering intricate real-world challenges, fostering collaboration and coordination among agents, and unleashing the potential for intelligent decision-making across domains. However, training a multi-agent reinforcement learning network is a formidable endeavor, demanding substantial computational resources to interact with diverse environmental variables, extract state representations, and acquire decision-making knowledge. The recent breakthroughs in large-scale pre-trained models ignite our curiosity: Can we uncover shared knowledge in multi-agent reinforcement learning and leverage pre-trained models to expedite training for future tasks? Addressing this issue, we present an innovative multi-task learning approach that aims to extract and harness common decision-making knowledge, like cooperation and competition, across different tasks. Our approach involves concurrent training of multiple multi-agent tasks, with each task employing independent front-end perception layers while sharing back-end decision-making layers. This effective decoupling of state representation extraction from decision-making allows for more efficient training and better transferability. To evaluate the efficacy of our proposed approach, we conduct comprehensive experiments in two distinct environments: the StarCraft Multi-agent Challenge (SMAC) and the Google Research Football (GRF) environments. The experimental results unequivocally demonstrate the smooth transferability of the shared decision-making network to other tasks, thereby significantly reducing training costs and improving final performance. Furthermore, visualizations authenticate the presence of general multi-agent decision-making knowledge within the shared network layers, further validating the effectiveness of our approach.

Meta-Learning Multi-task Communication.

Multi-task Learning with Gradient Communication

Meta Multi-Task Learning for Sequence Modeling.

Sharing to Learn and Learning to Share; Fitting Together Meta, Multi-Task, and Transfer Learning: A Meta Review

Sharing to learn and learning to share; Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta review

Learning Multi-Task Communication with Message Passing for Sequence Learning.

A Task-Aware Attention-Based Method for Improved Meta-Learning.

Multi-Task Learning in Natural Language Processing: An Overview

Multiple Relational Attention Network for Multi-task Learning

Informative Relationship Multi-Task Learning: Exploring Pairwise Contribution Across Tasks' Sharing Knowledge

Hacking Task Confounder in Meta-Learning

Multi-Task Networks With Universe, Group, and Task Feature Learning

Learning Multiple Tasks with Multilinear Relationship Networks.

Deep Multi-Task Learning with Shared Memory

A Meta Understanding of Meta-Learning

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

Multi-Task Reinforcement Learning with Attention-Based Mixture of Experts.

Data-Efficient Brain Connectome Analysis via Multi-Task Meta-Learning

Multi-Task Multi-Agent Shared Layers are Universal Cognition of Multi-Agent Coordination

Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons