Abstract:The game of Codenames has recently emerged as a domain of interest for intelligent agent design. The game is unique due to the way that language and coordination between teammates play important roles. Previous approaches to designing agents for this game have utilized a single internal language model to determine action choices. This often leads to good performance with some teammates and inferior performance with other teammates, as the agent cannot adapt to any specific teammate. In this paper we present the first adaptive agent for playing Codenames. We adopt an ensemble approach with the goal of determining, during the course of interacting with a specific teammate, which of our internal expert agents, each potentially with its own language model, is the best match. One difficulty faced in this approach is the lack of a single numerical metric that accurately captures the performance of a Codenames team. Prior Codenames research has utilized a handful of different metrics to evaluate agent teams. We propose a novel single metric to evaluate the performance of a Codenames team, whether playing a single team (solitaire) game, or a competitive game against another team. We then present and analyze an ensemble agent which selects an internal expert on each turn in order to maximize this proposed metric. Experimental analysis shows that this ensemble approach adapts to individual teammates and often performs nearly as well as the best internal expert with a teammate. Crucially, this success does not depend on any previous knowledge about the teammates, the ensemble agents, or their compatibility. This research represents an important step to making language-based agents for cooperative language settings like Codenames more adaptable to individual teammates.

Communicate to Play: Pragmatic Reasoning for Efficient Cross-Cultural Communication in Codenames

Modeling Cross-Cultural Pragmatic Inference with Codenames Duet

Learning to Mediate Disparities Towards Pragmatic Communication

Pragmatic Reasoning in Structured Signaling Games

Incorporating Pragmatic Reasoning Communication into Emergent Language

Learning to refer informatively by amortizing pragmatic reasoning

CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents

Emergent Communication for Rules Reasoning

Multi-dimensional reasoning in competitive resource allocation games: Evidence from intra-team communication

Towards automating Codenames spymasters with deep reinforcement learning

Improving Cooperation in Language Games with Bayesian Inference and the Cognitive Hierarchy

Playing Codenames with Language Graphs and Word Embeddings

Planning, Inference and Pragmatics in Sequential Language Games

Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text

Pragmatic inference and visual abstraction enable contextual flexibility during visual communication

Pow-Wow: A Dataset and Study on Collaborative Communication in Pommerman

Human-Agent Cooperation in Games under Incomplete Information through Natural Language Communication

Reasoning About Pragmatics with Neural Listeners and Speakers

Adapting to Teammates in a Cooperative Language Game

Universal Principles of Human Communication: Preliminary Evidence From a Cross-cultural Communication Game

Scalable pragmatic communication via self-supervision