Abstract:The emergent reasoning and Theory of Mind (ToM) abilities demonstrated by Large Language Models (LLMs) make them promising candidates for developing coordination agents. In this study, we introduce a new LLM-Coordination Benchmark aimed at a detailed analysis of LLMs within the context of Pure Coordination Games, where participating agents need to cooperate for the most gain. This benchmark evaluates LLMs through two distinct tasks: (1) \emph{Agentic Coordination}, where LLMs act as proactive participants for cooperation in 4 pure coordination games; (2) \emph{Coordination Question Answering (QA)}, where LLMs are prompted to answer 198 multiple-choice questions from the 4 games for evaluation of three key reasoning abilities: Environment Comprehension, ToM Reasoning, and Joint Planning. Furthermore, to enable LLMs for multi-agent coordination, we introduce a Cognitive Architecture for Coordination (CAC) framework that can easily integrate different LLMs as plug-and-play modules for pure coordination games. Our findings indicate that LLM agents equipped with GPT-4-turbo achieve comparable performance to state-of-the-art reinforcement learning methods in games that require commonsense actions based on the environment. Besides, zero-shot coordination experiments reveal that, unlike RL methods, LLM agents are robust to new unseen partners. However, results on Coordination QA show a large room for improvement in the Theory of Mind reasoning and joint planning abilities of LLMs. The analysis also sheds light on how the ability of LLMs to understand their environment and their partner's beliefs and intentions plays a part in their ability to plan for coordination. Our code is available at \url{<a class="link-external link-https" href="https://github.com/eric-ai-lab/llm_coordination" rel="external noopener nofollow">this https URL</a>}.

Research on A Kind of Coordination Machine Learning Model Based on The DFS

A Multi-Agent Learning Model Based on Dynamic Fuzzy Logic.

Learning Robust Skills for Tightly Coordinated Arms in Contact-Rich Tasks

Model inference of a dynamic system by fuzzy learning of geometric structures

Coordination as inference in multi-agent reinforcement learning

Learning control strategy based on human-machine on-line cooperation

Agent intelligence model based on dynamic fuzzy logic

A Cooperative Multi-Agent Reinforcement Learning Method Based on Coordination Degree

Analysis of deep learning technique using a complex spherical fuzzy rough decision support model

Coordination Graph-Based Deep Reinforcement Learning for Cooperative Spectrum Sensing under Correlated Fading

Context-Aware Sparse Deep Coordination Graphs

On Human Machine Cooperative Learning Control

Cooperative Open-ended Learning Framework for Zero-shot Coordination

Meta Neural Coordination

C3F: Constant Collaboration and Communication Framework for Graph-Representation Dynamic Multi-Robotic Systems

Research on the coordination algorithms in multiagent systems for automatic abstracting

DeepMAG: Deep reinforcement learning with multi-agent graphs for flexible job shop scheduling

Research on decision-level fusion method based on structural causal model in system-level fault detection and diagnosis

Dynamic Resource Management for Federated Edge Learning With Imperfect CSI: A Deep Reinforcement Learning Approach

LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models

A machine learning framework for solving high-dimensional mean field game and mean field control problems