Abstract:Existing agents based on large language models (LLMs) demonstrate robust problem-solving capabilities by integrating LLMs' inherent knowledge, strong in-context learning and zero-shot capabilities, and the use of tools combined with intricately designed LLM invocation workflows by humans. However, these agents still exhibit shortcomings in long-term reasoning and under-use the potential of existing tools, leading to noticeable deficiencies in complex real-world reasoning scenarios. To address these limitations, we introduce Sibyl, a simple yet powerful LLM-based agent framework designed to tackle complex reasoning tasks by efficiently leveraging a minimal set of tools. Drawing inspiration from Global Workspace Theory, Sibyl incorporates a global workspace to enhance the management and sharing of knowledge and conversation history throughout the system. Furthermore, guided by Society of Mind Theory, Sibyl implements a multi-agent debate-based jury to self-refine the final answers, ensuring a comprehensive and balanced approach. This approach aims to reduce system complexity while expanding the scope of problems solvable-from matters typically resolved by humans in minutes to those requiring hours or even days, thus facilitating a shift from System-1 to System-2 thinking. Sibyl has been designed with a focus on scalability and ease of debugging by incorporating the concept of reentrancy from functional programming from its inception, with the aim of seamless and low effort integration in other LLM applications to improve capabilities. Our experimental results on the GAIA benchmark test set reveal that the Sibyl agent instantiated with GPT-4 achieves state-of-the-art performance with an average score of 34.55%, compared to other agents based on GPT-4. We hope that Sibyl can inspire more reliable and reusable LLM-based agent solutions to address complex real-world reasoning tasks.

Sibilla: A Tool for Reasoning about Collective Systems

Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning

Pacti: Scaling Assume-Guarantee Reasoning for System Analysis and Design

Hybrid Automated Reasoning Tools: from Black-box to Clear-box Integration

Tool interoperability for model-based systems engineering

A Formal Approach to the Engineering of Domain-Specific Distributed Systems

Intelligence Benevolent Tools: A Global System Automating Integration of Structured and Semistructured Sources in One Process

Supporting heterogeneous distributed substantiation of common generic structures in collaborative design

SoS contract verification using statistical model checking

rΣ: Automated reasoning tool for non-functional requirement goal models

SIBILA: A novel interpretable ensemble of general-purpose machine learning models applied to medical contexts

Semantic Integration of Bosch Manufacturing Data Using Virtual Knowledge Graphs

Global Intelligence Benevolent Builder (GIBB): a System Automating Integration of Heterogeneous Classical Databases and Web

A Design Tool to Reason about Ambient Assisted Living Systems

An Open Challenge Problem Repository for Systems Supporting Binders

Systematic Evaluation and Usability Analysis of Formal Tools for Railway System Design

Reasoning support for risk prediction and prevention in independent living

A Tool Integration Language to Formalize Co-simulation Tool-Chains for Cyber-Physical System (CPS).

Integration of Formal Proof into Unified Assurance Cases with Isabelle/SACM

Exploring Collective DSL Integration in a Large Situated IS: Towards Comprehensive Language Integration in Information Systems

Intelligent Binary Schema Matching System in Heterogeneity Environment