Abstract:Multi-agent reinforcement learning (MARL) has received increasing attention for its applications in various domains. Researchers have paid much attention on its partially observable and cooperative settings for meeting real-world requirements. For testing performance of different algorithms, standardized environments are designed such as the StarCraft Multi-Agent Challenge, which is one of the most successful MARL benchmarks. To our best knowledge, most of current environments are synchronous, where agents execute actions in the same pace. However, heterogeneous agents usually have their own action spaces and there is no guarantee for actions from different agents to have the same executed cycle, which leads to asynchronous multi-agent cooperation. Inspired from the Wargame, a confrontation game between two armies abstracted from real world environment, we propose the first Partially Observable Asynchronous multi-agent Cooperation challenge (POAC) for the MARL community. Specifically, POAC supports two teams of heterogeneous agents to fight with each other, where an agent selects actions based on its own observations and cooperates asynchronously with its allies. Moreover, POAC is a light weight, flexible and easy to use environment, which can be configured by users to meet different experimental requirements such as self-play model, human-AI model and so on. Along with our benchmark, we offer six game scenarios of varying difficulties with the built-in rule-based AI as opponents. Finally, since most MARL algorithms are designed for synchronous agents, we revise several representatives to meet the asynchronous setting, and the relatively poor experimental results validate the challenge of POAC. Source code is released in \url{<a class="link-external link-http" href="http://turingai.ia.ac.cn/data" rel="external noopener nofollow">this http URL</a>\_center/show}.

Unlocking the Potential of MAPPO with Asynchronous Optimization

MAPPO method based on attention behavior network

Assigning Credit with Partial Reward Decoupling in Multi-Agent Proximal Policy Optimization

The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games

The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games

B2MAPO: A Batch-by-Batch Multi-Agent Policy Optimization to Balance Performance and Efficiency

Off-Policy Multi-Agent Decomposed Policy Gradients

Policy Regularization via Noisy Advantage Values for Cooperative Multi-agent Actor-Critic methods

Optimistic Multi-Agent Policy Gradient

Coordinated Proximal Policy Optimization

Boosting Weak-to-Strong Agents in Multiagent Reinforcement Learning via Balanced PPO

Communication-Efficient Cooperative Multi-Agent PPO via Regulated Segment Mixture in Internet of Vehicles

FP3O: Enabling Proximal Policy Optimization in Multi-Agent Cooperation with Parameter-Sharing Versatility

Meta Proximal Policy Optimization for Cooperative Multi-Agent Continuous Control

The Partially Observable Asynchronous Multi-Agent Cooperation Challenge

JointPPO: Diving Deeper into the Effectiveness of PPO in Multi-Agent Reinforcement Learning

Scalable Model-based Policy Optimization for Decentralized Networked Systems

Intelligent Decentralized Multiple Access Via Multi- Agent Deep Reinforcement Learning

Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts

Proximal Policy Optimization with Future Rewards

An Improved PPO for Multiple Unmanned Aerial Vehicles