Abstract:In an era where single large language models have dominated the landscape of artificial intelligence for years, multi-agent systems arise as new protagonists in conversational task-solving. While previous studies have showcased their potential in reasoning tasks and creative endeavors, an analysis of their limitations concerning the conversational paradigms and the impact of individual agents is missing. It remains unascertained how multi-agent discussions perform across tasks of varying complexity and how the structure of these conversations influences the process. To fill that gap, this work systematically evaluates multi-agent systems across various discussion paradigms, assessing their strengths and weaknesses in both generative tasks and question-answering tasks. Alongside the experiments, I propose a taxonomy of 20 multi-agent research studies from 2022 to 2024, followed by the introduction of a framework for deploying multi-agent LLMs in conversational task-solving. I demonstrate that while multi-agent systems excel in complex reasoning tasks, outperforming a single model by leveraging expert personas, they fail on basic tasks. Concretely, I identify three challenges that arise: 1) While longer discussions enhance reasoning, agents fail to maintain conformity to strict task requirements, which leads to problem drift, making shorter conversations more effective for basic tasks. 2) Prolonged discussions risk alignment collapse, raising new safety concerns for these systems. 3) I showcase discussion monopolization through long generations, posing the problem of fairness in decision-making for tasks like summarization. This work uncovers both the potential and challenges that arise with multi-agent interaction and varying conversational paradigms, providing insights into how future research could improve the efficiency, performance, and safety of multi-agent LLMs.

Multi-Agent Collaboration in Incident Response with Large Language Models

Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game

MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate

Theory of Mind for Multi-Agent Collaboration via Large Language Models

Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration

Synergistic Simulations: Multi-Agent Problem Solving with Large Language Models

Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents

Large Language Model based Multi-Agents: A Survey of Progress and Challenges

MHRC: Closed-loop Decentralized Multi-Heterogeneous Robot Collaboration with Large Language Models

Embodied LLM Agents Learn to Cooperate in Organized Teams

Challenges Faced by Large Language Models in Solving Multi-Agent Flocking

Large Language Model-based Human-Agent Collaboration for Complex Task Solving

Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents

MindAgent: Emergent Gaming Interaction

Multi-Agent Large Language Models for Conversational Task-Solving

Leveraging Large Language Model for Heterogeneous Ad Hoc Teamwork Collaboration

Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information

Large Language Models in Cybersecurity: State-of-the-Art

Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games