Abstract:Multi-Agent Reinforcement Learning (MARL) is extensively utilized for addressing intricate tasks that involve cooperation and competition among agents in Multi-Agent Systems (MAS). However, learning such tasks from scratch is challenging and often unfeasible, especially for MASs with a large number of agents. Hence, leveraging knowledge from prior experiences can effectively expedite the MARL learning process. Prior work has shown that we successfully facilitated transfer learning for MARL by consolidating various state spaces into fixed-size inputs, enabling a single unified deep-learning policy applicable to several scenarios within the StarCraft Multi-Agent Challenge (SMAC) environment. In this study, we expand SMAC to Multi-Player enabled SMAC (MP-SMAC) by enabling the dynamic selection of training opponents and introducing a co-evolving MARL framework, which creates a co-evolutionary arena where multiple policies learn simultaneously. Our arena comprised the simultaneous training of multiple policies in diverse scenarios, pitting them against both static AI opponents and their peers within MP-SMAC. Furthermore, we integrate co-evolution with curriculum transfer learning into Co-MACTRL framework, enabling our MARL policies to systematically acquire knowledge and skills across predetermined scenarios organized by varying difficulty levels, including evolving opponents. The results revealed significant enhancements in MARL learning performance, demonstrating the advantage of leveraging the co-evolving opponents and maneuvering skills obtained from different scenarios. Additionally, the Co-MACTRL learners consistently attained high performance across a range of SMAC scenarios, showcasing the robustness and generalizability of Co-MACTRL.

Continuous Policy Multi-Agent Deep Reinforcement Learning with Generalizable Episodic Memory

State-based episodic memory for multi-agent reinforcement learning

Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learning

Deep Reinforcement Learning with Parametric Episodic Memory

Generalizable Episodic Memory for Deep Reinforcement Learning

A Graph-Based Soft Actor Critic Approach in Multi-Agent Reinforcement Learning

Episodic Reinforcement Learning with Associative Memory.

Off-Beat Multi-Agent Reinforcement Learning

Episodic Multi-agent Reinforcement Learning with Curiosity-driven Exploration

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Episodic Reinforcement Learning with Expanded State-reward Space

Continuous Episodic Control

Multiexperience-Assisted Efficient Multiagent Reinforcement Learning

Two-Memory Reinforcement Learning

Decentralized Multi-Agent Reinforcement Learning: An Off-Policy Method

Co-Evolving Multi-Agent Transfer Reinforcement Learning Via Scenario Independent Representation

Multi-agent Continual Coordination Via Progressive Task Contextualization

Sample-Efficient Multiagent Reinforcement Learning with Reset Replay

Multiagent Continual Coordination via Progressive Task Contextualization

ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency

Hierarchical Deep Multiagent Reinforcement Learning with Temporal Abstraction