Abstract:Training agents that can coordinate zero-shot with humans is a key mission in multi-agent reinforcement learning (MARL). Current algorithms focus on training simulated human partner policies which are then used to train a Cooperator agent. The simulated human is produced either through behavior cloning over a dataset of human cooperation behavior, or by using MARL to create a population of simulated agents. However, these approaches often struggle to produce a Cooperator that can coordinate well with real humans, since the simulated humans fail to cover the diverse strategies and styles employed by people in the real world. We show \emph{learning a generative model of human partners} can effectively address this issue. Our model learns a latent variable representation of the human that can be regarded as encoding the human's unique strategy, intention, experience, or style. This generative model can be flexibly trained from any (human or neural policy) agent interaction data. By sampling from the latent space, we can use the generative model to produce different partners to train Cooperator agents. We evaluate our method -- \textbf{G}enerative \textbf{A}gent \textbf{M}odeling for \textbf{M}ulti-agent \textbf{A}daptation (GAMMA) -- on Overcooked, a challenging cooperative cooking game that has become a standard benchmark for zero-shot coordination. We conduct an evaluation with real human teammates, and the results show that GAMMA consistently improves performance, whether the generative model is trained on simulated populations or human datasets. Further, we propose a method for posterior sampling from the generative model that is biased towards the human data, enabling us to efficiently improve performance with only a small amount of expensive human interaction data.

Towards Controllable Agent in MOBA Games with Generative Modeling

Towards Effective and Interpretable Human-Agent Collaboration in MOBA Games: A Communication Perspective

Hierarchical Reinforcement Learning for Multi-agent MOBA Game

Towards Playing Full MOBA Games with Deep Reinforcement Learning

Enhancing Human Experience in Human-Agent Collaboration: A Human-Centered Modeling Approach Based on Positive Human Gain

Mastering Complex Control in MOBA Games with Deep Reinforcement Learning

Hierarchical Macro Strategy Model for MOBA Game AI

Learning to Cooperate with Humans using Generative Agents

Learning Latent Representations to Influence Multi-Agent Interaction

A Machine Learning Approach to Detect Strategic Behavior from Large-Population Observational Data Applied to Game Mode Prediction on a Team-Based Video Game

ProAgent: Building Proactive Cooperative Agents with Large Language Models

Toward Human-AI Alignment in Large-Scale Multi-Player Games

Grounded Answers for Multi-agent Decision-making Problem through Generative World Model

A Controllable Co-Creative Agent for Game System Design

Modeling Theory of Mind in Multi-Agent Games Using Adaptive Feedback Control

Supervised Learning Achieves Human-Level Performance in MOBA Games: A Case Study of Honor of Kings

Cooperative multi-agent game based on reinforcement learning

Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

Modeling Game Avatar Synergy and Opposition through Embedding in Multiplayer Online Battle Arena Games

Aligning Individual and Collective Objectives in Multi-Agent Cooperation