Abstract:Multi-agent reinforcement learning (MARL) has received increasing attention for its applications in various domains. Researchers have paid much attention on its partially observable and cooperative settings for meeting real-world requirements. For testing performance of different algorithms, standardized environments are designed such as the StarCraft Multi-Agent Challenge, which is one of the most successful MARL benchmarks. To our best knowledge, most of current environments are synchronous, where agents execute actions in the same pace. However, heterogeneous agents usually have their own action spaces and there is no guarantee for actions from different agents to have the same executed cycle, which leads to asynchronous multi-agent cooperation. Inspired from the Wargame, a confrontation game between two armies abstracted from real world environment, we propose the first Partially Observable Asynchronous multi-agent Cooperation challenge (POAC) for the MARL community. Specifically, POAC supports two teams of heterogeneous agents to fight with each other, where an agent selects actions based on its own observations and cooperates asynchronously with its allies. Moreover, POAC is a light weight, flexible and easy to use environment, which can be configured by users to meet different experimental requirements such as self-play model, human-AI model and so on. Along with our benchmark, we offer six game scenarios of varying difficulties with the built-in rule-based AI as opponents. Finally, since most MARL algorithms are designed for synchronous agents, we revise several representatives to meet the asynchronous setting, and the relatively poor experimental results validate the challenge of POAC. Source code is released in \url{<a class="link-external link-http" href="http://turingai.ia.ac.cn/data" rel="external noopener nofollow">this http URL</a>\_center/show}.

Arena: a toolkit for Multi-Agent Reinforcement Learning

Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence

S2rl

FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning

DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation

Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?

TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

CaiRL: A High-Performance Reinforcement Learning Environment Toolkit

NeuronsMAE: A Novel Multi-Agent Reinforcement Learning Environment for Cooperative and Competitive Multi-Robot Tasks

Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learning

SC-MAIRL: Semi-Centralized Multi-Agent Imitation Reinforcement Learning

Arena 3.0: Advancing Social Navigation in Collaborative and Highly Dynamic Environments

The Arcade Learning Environment: An Evaluation Platform for General Agents

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search

MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library

The Partially Observable Asynchronous Multi-Agent Cooperation Challenge

Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning

ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency

PantheonRL: A MARL Library for Dynamic Training Interactions