OpenAGI: When LLM Meets Domain Experts

Yingqiang Ge,Wenyue Hua,Kai Mei,Jianchao Ji,Juntao Tan,Shuyuan Xu,Zelong Li,Yongfeng Zhang

2023-11-03

Abstract:Human Intelligence (HI) excels at combining basic skills to solve complex tasks. This capability is vital for Artificial Intelligence (AI) and should be embedded in comprehensive AI Agents, enabling them to harness expert models for complex task-solving towards Artificial General Intelligence (AGI). Large Language Models (LLMs) show promising learning and reasoning abilities, and can effectively use external models, tools, plugins, or APIs to tackle complex problems. In this work, we introduce OpenAGI, an open-source AGI research and development platform designed for solving multi-step, real-world tasks. Specifically, OpenAGI uses a dual strategy, integrating standard benchmark tasks for benchmarking and evaluation, and open-ended tasks including more expandable models, tools, plugins, or APIs for creative problem-solving. Tasks are presented as natural language queries to the LLM, which then selects and executes appropriate models. We also propose a Reinforcement Learning from Task Feedback (RLTF) mechanism that uses task results to improve the LLM's task-solving ability, which creates a self-improving AI feedback loop. While we acknowledge that AGI is a broad and multifaceted research challenge with no singularly defined solution path, the integration of LLMs with domain-specific expert models, inspired by mirroring the blend of general and specialized intelligence in humans, offers a promising approach towards AGI. We are open-sourcing the OpenAGI project's code, dataset, benchmarks, evaluation methods, and the UI demo to foster community involvement in AGI advancement: <a class="link-external link-https" href="https://github.com/agiresearch/OpenAGI" rel="external noopener nofollow">this https URL</a>.

Artificial Intelligence,Computation and Language,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the issue of how to build an Artificial General Intelligence (AGI) platform capable of handling complex, multi-step tasks. Specifically, the paper proposes an open-source research and development platform called OpenAGI, which combines Large Language Models (LLMs) with domain-specific expert models to solve complex real-world problems.

To achieve this goal, OpenAGI employs the following strategies and techniques:

1. **Integration of Standard Benchmark Tasks**: The platform includes a series of standard benchmark tasks along with their corresponding datasets and evaluation metrics to assess the performance of different models and facilitate their continuous improvement.

2. **Open-ended Tasks**: In addition to benchmark tasks, open-ended tasks are provided, allowing for the use of more scalable models and tools for creative problem-solving.

3. **Natural Language Instruction Processing**: Complex tasks are presented to the LLM in the form of natural language, and the LLM generates a plan to solve the problem.

4. **Model Synthesis and Execution**: Appropriate expert models are selected and executed based on the generated plan to handle data samples.

5. **Self-Improvement Mechanism**: A Reinforcement Learning from Task Feedback (RLTF) mechanism is proposed, utilizing task results to improve the LLM's task-solving capabilities, thereby forming a self-improving AI feedback loop.

6. **Dual-Strategy Approach**: A combination of benchmark tasks and open-ended tasks is used to meet diverse research needs.

The paper also discusses the challenges faced in the current field, such as scalability, non-linear task planning, and quantitative evaluation, and introduces how the OpenAGI platform can mitigate these issues. Additionally, the paper emphasizes the importance of community involvement and has open-sourced the project's code, datasets, benchmarks, evaluation methods, and demonstration interface to facilitate the community's collective advancement of AGI.