Abstract:Existing agents based on large language models (LLMs) demonstrate robust problem-solving capabilities by integrating LLMs' inherent knowledge, strong in-context learning and zero-shot capabilities, and the use of tools combined with intricately designed LLM invocation workflows by humans. However, these agents still exhibit shortcomings in long-term reasoning and under-use the potential of existing tools, leading to noticeable deficiencies in complex real-world reasoning scenarios. To address these limitations, we introduce Sibyl, a simple yet powerful LLM-based agent framework designed to tackle complex reasoning tasks by efficiently leveraging a minimal set of tools. Drawing inspiration from Global Workspace Theory, Sibyl incorporates a global workspace to enhance the management and sharing of knowledge and conversation history throughout the system. Furthermore, guided by Society of Mind Theory, Sibyl implements a multi-agent debate-based jury to self-refine the final answers, ensuring a comprehensive and balanced approach. This approach aims to reduce system complexity while expanding the scope of problems solvable-from matters typically resolved by humans in minutes to those requiring hours or even days, thus facilitating a shift from System-1 to System-2 thinking. Sibyl has been designed with a focus on scalability and ease of debugging by incorporating the concept of reentrancy from functional programming from its inception, with the aim of seamless and low effort integration in other LLM applications to improve capabilities. Our experimental results on the GAIA benchmark test set reveal that the Sibyl agent instantiated with GPT-4 achieves state-of-the-art performance with an average score of 34.55%, compared to other agents based on GPT-4. We hope that Sibyl can inspire more reliable and reusable LLM-based agent solutions to address complex real-world reasoning tasks.

What problem does this paper attempt to address?

The paper aims to address the issues faced by agents based on large language models (LLMs) when handling complex real-world reasoning tasks. Specifically, although existing LLM agents demonstrate strong problem-solving capabilities, they still fall short in long-term reasoning and fail to fully utilize the potential of available tools. This results in significant shortcomings when dealing with complex real-world reasoning scenarios. To address these issues, the authors propose the Sibyl framework, a simple yet powerful LLM-based agent framework designed to tackle complex reasoning tasks by efficiently utilizing a minimal number of tools. Sibyl draws inspiration from the Global Workspace Theory, introducing a global workspace to enhance knowledge and dialogue history management throughout the system. Additionally, guided by the Society of Mind Theory, Sibyl implements a jury mechanism based on multi-agent debate to self-correct final answers, ensuring comprehensiveness and balance in the approach. This method aims to reduce system complexity while expanding the range of solvable problems, thereby facilitating a shift from fast, intuitive System 1 thinking to slow, deliberative System 2 thinking.

Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning

S2rl

Sibyl: Empowering Empathetic Dialogue Generation in Large Language Models via Sensible and Visionary Commonsense Inference

SocraSynth: Multi-LLM Reasoning with Conditional Statistics

SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks

KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate

ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration

XAgents: A Framework for Interpretable Rule-Based Multi-Agents Cooperation

STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making

Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

SciAgent: Tool-augmented Language Models for Scientific Reasoning

A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration

DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning

Multi-Agent Large Language Models for Conversational Task-Solving

AgentKit: Structured LLM Reasoning with Dynamic Graphs

ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Data

Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation