Abstract:Securing coordination between AI agent and teammates (human players or AI agents) in contexts involving unfamiliar humans continues to pose a significant challenge in Zero-Shot Coordination. The issue of cooperative incompatibility becomes particularly prominent when an AI agent is unsuccessful in synchronizing with certain previously unknown partners. Traditional algorithms have aimed to collaborate with partners by optimizing fixed objectives within a population, fostering diversity in strategies and behaviors. However, these techniques may lead to learning loss and an inability to cooperate with specific strategies within the population, a phenomenon named cooperative incompatibility in learning. In order to solve cooperative incompatibility in learning and effectively address the problem in the context of ZSC, we introduce the Cooperative Open-ended LEarning (COLE) framework, which formulates open-ended objectives in cooperative games with two players using perspectives of graph theory to evaluate and pinpoint the cooperative capacity of each strategy. We present two practical algorithms, specifically \algo and \algoR, which incorporate insights from game theory and graph theory. We also show that COLE could effectively overcome the cooperative incompatibility from theoretical and empirical analysis. Subsequently, we created an online Overcooked human-AI experiment platform, the COLE platform, which enables easy customization of questionnaires, model weights, and other aspects. Utilizing the COLE platform, we enlist 130 participants for human experiments. Our findings reveal a preference for our approach over state-of-the-art methods using a variety of subjective metrics. Moreover, objective experimental outcomes in the Overcooked game environment indicate that our method surpasses existing ones when coordinating with previously unencountered AI agents and the human proxy model.

Maximum Entropy Population Based Training for Zero-Shot Human-AI Coordination

Collaborating with Humans without Human Data

Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biased

PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination

"Other-Play" for Zero-Shot Coordination

Tackling Cooperative Incompatibility for Zero-Shot Human-AI Coordination

Learning to Cooperate with Humans using Generative Agents

Any-Play: An Intrinsic Augmentation for Zero-Shot Coordination

Human-AI Coordination via Human-Regularized Search and Learning

KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordination

Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates

Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Human-compatible driving partners through data-regularized self-play reinforcement learning

Role Play: Learning Adaptive Role-Specific Strategies in Multi-Agent Interactions

Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi

Improving Zero-Shot Coordination Performance Based on Policy Similarity

Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaboration

Cooperative Open-ended Learning Framework for Zero-shot Coordination

Coordination with Humans via Strategy Matching

Multi-Expert Distillation for Few-Shot Coordination (student Abstract)