AIOS: LLM Agent Operating System

Kai Mei,Zelong Li,Shuyuan Xu,Ruosong Ye,Yingqiang Ge,Yongfeng Zhang
2024-03-26
Abstract:The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS) as the brain of the OS, enabling an operating system "with soul" -- an important step towards AGI. Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future. The project is open-source at
Operating Systems,Artificial Intelligence,Computation and Language
What problem does this paper attempt to address?
The problem addressed in this paper is how to effectively integrate and deploy intelligent agents based on large-scale language models (LLMs) to solve issues such as suboptimal resource allocation, difficulties in maintaining context, complex integration of heterogeneous agents, and bottlenecks and low resource utilization caused by increasing agent quantity and complexity. The paper introduces AIOS (LLM Agent Operating System), which is an operating system that embeds large-scale language models as its core, aiming to optimize resource allocation, simplify context switching, support concurrent execution of agents, provide tool services, and maintain access control. AIOS designs a dedicated kernel for LLMs to isolate and aggregate the functionalities of LLMs and the operating system, solving possible conflicts between LLM and non-LLM tasks and managing activities related to LLMs. Through experiments, the paper demonstrates the reliability and efficiency of AIOS in concurrent execution of multiple agents. The objective is to enhance the performance and efficiency of LLM agents and pave the way for future development and deployment of a better AIOS ecosystem. The project is open-source and can be found at https://github.com/agiresearch/AIOS.