Abstract:Securing coordination between AI agent and teammates (human players or AI agents) in contexts involving unfamiliar humans continues to pose a significant challenge in Zero-Shot Coordination. The issue of cooperative incompatibility becomes particularly prominent when an AI agent is unsuccessful in synchronizing with certain previously unknown partners. Traditional algorithms have aimed to collaborate with partners by optimizing fixed objectives within a population, fostering diversity in strategies and behaviors. However, these techniques may lead to learning loss and an inability to cooperate with specific strategies within the population, a phenomenon named cooperative incompatibility in learning. In order to solve cooperative incompatibility in learning and effectively address the problem in the context of ZSC, we introduce the Cooperative Open-ended LEarning (COLE) framework, which formulates open-ended objectives in cooperative games with two players using perspectives of graph theory to evaluate and pinpoint the cooperative capacity of each strategy. We present two practical algorithms, specifically \algo and \algoR, which incorporate insights from game theory and graph theory. We also show that COLE could effectively overcome the cooperative incompatibility from theoretical and empirical analysis. Subsequently, we created an online Overcooked human-AI experiment platform, the COLE platform, which enables easy customization of questionnaires, model weights, and other aspects. Utilizing the COLE platform, we enlist 130 participants for human experiments. Our findings reveal a preference for our approach over state-of-the-art methods using a variety of subjective metrics. Moreover, objective experimental outcomes in the Overcooked game environment indicate that our method surpasses existing ones when coordinating with previously unencountered AI agents and the human proxy model.

Open and Real-World Human-Ai Coordination by Heterogeneous Training with Communication

Tackling Cooperative Incompatibility for Zero-Shot Human-AI Coordination

PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination

Efficient Human-AI Coordination via Preparatory Language-based Convention

CREW: Facilitating Human-AI Teaming Research

Open Human-Robot Collaboration using Decentralized Inverse Reinforcement Learning

Human-AI Coordination via Human-Regularized Search and Learning

Open Ad Hoc Teamwork with Cooperative Game Theory

Cooperative Open-ended Learning Framework for Zero-shot Coordination

Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning

Leveraging Large Language Model for Heterogeneous Ad Hoc Teamwork Collaboration

CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics

Maximum Entropy Population Based Training for Zero-Shot Human-AI Coordination

Heterogeneous Policy Networks for Composite Robot Team Communication and Coordination

Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution

Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

Demonstrating HumanTHOR: A Simulation Platform and Benchmark for Human-Robot Collaboration in a Shared Workspace

CoHRT: A Collaboration System for Human-Robot Teamwork

Coordination Scheme Probing for Generalizable Multi-Agent Reinforcement Learning

Effect of Adaptive Communication Support on Human-AI Collaboration