Abstract:Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy generalization depends heavily on the diversity of teammates they interact with during the training phase. Such reliance, however, constrains the agents' capacity for strategic adaptation when cooperating with unfamiliar teammates, which becomes a significant challenge in zero-shot coordination scenarios. To address this challenge, we propose ProAgent, a novel framework that harnesses large language models (LLMs) to create proactive agents capable of dynamically adapting their behavior to enhance cooperation with teammates. ProAgent can analyze the present state, and infer the intentions of teammates from observations. It then updates its beliefs in alignment with the teammates' subsequent actual behaviors. Moreover, ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various of coordination scenarios. Experimental evaluations conducted within the Overcooked-AI environment unveil the remarkable performance superiority of ProAgent, outperforming five methods based on self-play and population-based training when cooperating with AI agents. Furthermore, in partnered with human proxy models, its performance exhibits an average improvement exceeding 10% compared to the current state-of-the-art method. For more information about our project, please visit~\url{<a class="link-external link-https" href="https://pku-proagent.github.io" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

The paper aims to address the challenges faced by cooperative agents in multi-agent systems when collaborating with unfamiliar teammates, particularly in zero-shot coordination scenarios. Current learning-based methods train cooperative agents whose strategy generalization ability is limited when collaborating with unseen teammates. To solve this problem, the paper proposes a new framework called ProAgent. ProAgent leverages large language models (LLMs) to create agents that can actively adapt to the environment and efficiently collaborate with teammates. Specifically, ProAgent has the following features: 1. **Dynamic Adaptability**: ProAgent can analyze the current state and infer the teammate's intentions from observations, then update its beliefs based on the teammate's subsequent actual behavior. 2. **Modularity and Interpretability**: ProAgent is highly modular and interpretable, making it easy to integrate into various collaborative scenarios. 3. **Innovative Mechanisms**: ProAgent includes four key modules—the Planner, Verificator, Controller, and Memory module, as well as a Belief Correction mechanism. These components work together to enable ProAgent to predict the teammate's intentions, achieving adaptive cooperative reasoning and planning. Through a series of experimental evaluations in the Overcooked-AI environment, the paper demonstrates ProAgent's ability to cooperate with various types of AI teammates, especially its superior performance in zero-shot coordination scenarios. The experimental results show that ProAgent exhibits significant advantages compared to existing self-play and population-based training methods. In summary, the study contributes in three aspects: - Integrating large language models into the cooperative multi-agent domain and proposing the ProAgent framework to utilize the powerful reasoning and planning capabilities of LLMs. - Demonstrating that ProAgent can clearly analyze the current scenario, infer the teammate's intentions, and dynamically adjust its behavior based on this information. - Providing evidence through a series of experiments that ProAgent excels in collaborating with different types of teammates.

ProAgent: Building Proactive Cooperative Agents with Large Language Models

Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance

Agents: An Open-source Framework for Autonomous Language Agents

Adaptive In-conversation Team Building for Language Model Agents

MegaAgent: A Practical Framework for Autonomous Cooperation in Large-Scale LLM Agent Systems

MindAgent: Emergent Gaming Interaction

Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game

Building Cooperative Embodied Agents Modularly with Large Language Models

Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization

Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning

MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents

AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors

TrainerAgent: Customizable and Efficient Model Training Through LLM-Powered Multi-Agent System.

Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models

CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agents

Learning to Cooperate with Humans using Generative Agents

Interactive Agent Modeling by Learning to Probe

The Rise and Potential of Large Language Model Based Agents: A Survey

AgentGym: Evolving Large Language Model-based Agents across Diverse Environments

ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models

ProAgent: From Robotic Process Automation to Agentic Process Automation