Abstract:Training agents that can coordinate zero-shot with humans is a key mission in multi-agent reinforcement learning (MARL). Current algorithms focus on training simulated human partner policies which are then used to train a Cooperator agent. The simulated human is produced either through behavior cloning over a dataset of human cooperation behavior, or by using MARL to create a population of simulated agents. However, these approaches often struggle to produce a Cooperator that can coordinate well with real humans, since the simulated humans fail to cover the diverse strategies and styles employed by people in the real world. We show \emph{learning a generative model of human partners} can effectively address this issue. Our model learns a latent variable representation of the human that can be regarded as encoding the human's unique strategy, intention, experience, or style. This generative model can be flexibly trained from any (human or neural policy) agent interaction data. By sampling from the latent space, we can use the generative model to produce different partners to train Cooperator agents. We evaluate our method -- \textbf{G}enerative \textbf{A}gent \textbf{M}odeling for \textbf{M}ulti-agent \textbf{A}daptation (GAMMA) -- on Overcooked, a challenging cooperative cooking game that has become a standard benchmark for zero-shot coordination. We conduct an evaluation with real human teammates, and the results show that GAMMA consistently improves performance, whether the generative model is trained on simulated populations or human datasets. Further, we propose a method for posterior sampling from the generative model that is biased towards the human data, enabling us to efficiently improve performance with only a small amount of expensive human interaction data.

SocialInteractionGAN: Multi-person Interaction Sequence Generation

Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks

Learning a Generative Model for Multi‐Step Human‐Object Interactions from Videos

Interaction Mix and Match: Synthesizing Close Interaction using Conditional Hierarchical GAN with Multi-Hot Class Embedding

Multi-Level Sequence GAN for Group Activity Recognition

Interactive Generative Adversarial Networks for Facial Expression Generation in Dyadic Interactions

Human Action Generation with Generative Adversarial Networks

Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks

Autoregressive GAN for Semantic Unconditional Head Motion Generation

Nonverbal social behavior generation for social robots using end-to-end learning

InterGen: Diffusion-Based Multi-human Motion Generation Under Complex Interactions

Synthesizing Human Activity for Data Generation

Generative Modeling of Multimodal Multi-Human Behavior

Towards Practical Single-shot Motion Synthesis

Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networks

Automatic cinematography for body movement involved virtual communication

InteractNet: Social Interaction Recognition for Semantic-rich Videos

Learning to Cooperate with Humans using Generative Agents

GANimator: Neural Motion Synthesis from a Single Sequence

InterMask: 3D Human Interaction Generation via Collaborative Masked Modelling

Generating Social-Aware Locations for a Robot in a Human Group by Image Generation Using Adversarial Learning