AOMO - An AI-aided Optimizer for Microservices Orchestration.

Xue Leng,Tzung-Han Juang,Yan Chen,Han Liu

DOI: https://doi.org/10.1145/3342280.3342287

2019-01-01

Abstract: An abstract is not available.

What problem does this paper attempt to address?

Service-oriented open architecture for complex process system modeling and optimization

CHEN Tao,SHAO Zhijiang

DOI: https://doi.org/10.3321/j.issn:0438-1157.2005.12.019

2005-01-01

Abstract:The state of the art process modeling and optimization software was reviewed. Motivated by increasing demand on modeling and optimization of systems with large-scale and high complexity characteristics, a new application mode with service-oriented open architecture was proposed.ASCEND (advanced system for computations in engineering design) was chosen as the base for equation-oriented modeling. Its inherent structure, data flow, working mode and communication interface were redefined and reconstructed. With support from COM/DCOM/AUTOMATION technology, AscendServer was built as the core service providing modeling and optimization functionalities. Some key issues such as componentization, event handling and coordination between processes were discussed in details. Within this superstructure, modeling and optimization for large-scale complex systems could be integrated with more extensive data exchange, more flexible application mode and more adaptative capabilities. A data reconciliation system of water consumption balancing for a petrochemical company was given to verify the efficiency and functionalities of the new open structure service for modeling and optimization.
Towards Automated Inter-Service Authorization for Microservice Applications.

Xing Li,Yan Chen,Zhiqiang Lin

DOI: https://doi.org/10.1145/3342280.3342288

2019-01-01

Abstract: An abstract is not available.
How autonomy oriented computing (AOC) tackles a computationally hard optimization problem

Xiao-Feng Xie,Jiming Liu

DOI: https://doi.org/10.1145/1160633.1160747

2006-01-01

Abstract:The hard computational problems, such as the traveling salesman problem (TSP), are relevant to many tasks of practical interest, which normally can be well formalized but are difficult to solve. This paper presents an extended multiagent optimization system, called MAOSE, for supporting cooperative problem solving on a virtual landscape and achieving high-quality solution(s) by the self-organization of autonomous entities. The realization of an optimization algorithm then can be described in three parts: a) encode the representation of the problem, which provides the virtual landscape and possible auxiliary knowledge; b) construct the memory elements at the initialization stage; and c) design the generate-and-test behavior guided by the law of socially-biased individual learning, through tailoring to the domain structure. The implementation is demonstrated on the TSP in details. The extensive experimental results on real-world instances in TSPLIB show its efficiency as comparing to other algorithms.
Many-objective ant lion optimizer (MaOALO): A new many-objective optimizer with its engineering applications

Kanak Kalita,Sundaram B Pandya,Robert Čep,Pradeep Jangir,Laith Abualigah

DOI: https://doi.org/10.1016/j.heliyon.2024.e32911

IF: 3.776

2024-06-17

Heliyon

Abstract:Many-objective optimization (MaO) is an important aspect of engineering scenarios. In many-objective optimization algorithms (MaOAs), a key challenge is to strike a balance between diversity and convergence. MaOAs employs various tactics to either enhance selection pressure for better convergence and/or implements additional measures for sustaining diversity. With increase in number of objectives, the process becomes more complex, mainly due to challenges in achieving convergence during population selection. This paper introduces a novel Many-Objective Ant Lion Optimizer (MaOALO), featuring the widely-popular ant lion optimizer algorithm. This method utilizes reference point, niche preserve and information feedback mechanism (IFM), to enhance the convergence and diversity of the population. Extensive experimental tests on five real-world (RWMaOP1- RWMaOP5) optimization problems and standard problem classes, including MaF1-MaF15 (for 5, 9 and 15 objectives), DTLZ1-DTLZ7 (for 8 objectives) has been carried out. It is shown that MaOALO is superior compared to ARMOEA, NSGA-III, MaOTLBO, RVEA, MaOABC-TA, DSAE, RL-RVEA and MaOEA-IH algorithms in terms of GD, IGD, SP, SD, HV and RT metrics. The MaOALO source code is available at: https://github.com/kanak02/MaOALO.
Machine Learning Insides OptVerse AI Solver: Design Principles and Applications

Xijun Li,Fangzhou Zhu,Hui-Ling Zhen,Weilin Luo,Meng Lu,Yimin Huang,Zhenan Fan,Zirui Zhou,Yufei Kuang,Zhihai Wang,Zijie Geng,Yang Li,Haoyang Liu,Zhiwu An,Muming Yang,Jianshu Li,Jie Wang,Junchi Yan,Defeng Sun,Tao Zhong,Yong Zhang,Jia Zeng,Mingxuan Yuan,Jianye Hao,Jun Yao,Kun Mao

2024-01-17

Abstract:In an era of digital ubiquity, efficient resource management and decision-making are paramount across numerous industries. To this end, we present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI Solver, which aims to mitigate the scarcity of real-world mathematical programming instances, and to surpass the capabilities of traditional optimization techniques. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. Furthermore, we introduce a training framework leveraging augmentation policies to maintain solvers' utility in dynamic environments. Besides the data generation and augmentation, our proposed approaches also include novel ML-driven policies for personalized solver strategies, with an emphasis on applications like graph convolutional networks for initial basis selection and reinforcement learning for advanced presolving and cut selection. Additionally, we detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance. Compared with traditional solvers such as Cplex and SCIP, our ML-augmented OptVerse AI Solver demonstrates superior speed and precision across both established benchmarks and real-world scenarios, reinforcing the practical imperative and effectiveness of machine learning techniques in mathematical programming solvers.

Artificial Intelligence
An improved ant colony optimization algorithm for QoS-aware dynamic web service composition

Shanshan Zhao,Lei Wang,Lin Ma,Zepeng Wen

DOI: https://doi.org/10.1109/ICICEE.2012.531

2012-01-01

Abstract:A lot of work have been carried out to find the efficient algorithms for QoS-aware dynamic web service composition. The multi-objective multi-choice problem as an evolutive multi-dimension multi-objective 0-1 knapsack problem (MMKP) can be reduced into the web service composition with multiple QoS constraints, while many approaches based on ant colony optimization (AOC) have been introduced in order for solving MMKP. This paper presents a novel QoS-aware dynamic web service composition optimization algorithm with improved AOC. The essence of the algorithm is that the problem of dynamic web service selection with QoS global optimality is transformed into a multi-objective multi-choice QoS-aware web services composition optimization (an evolutive MMKP), for the final purpose of our algorithm is to achieve Pareto Optimality. © 2012 IEEE.
An enhanced hybrid arithmetic optimization algorithm for engineering applications

Gang Hu,Jingyu Zhong,Bo Du,Guo Wei

DOI: https://doi.org/10.1016/j.cma.2022.114901

IF: 6.588

2022-05-01

Computer Methods in Applied Mechanics and Engineering

Abstract:Arithmetic optimization algorithm (AOA) is a newly well-developed meta-heuristic algorithm that is inspired by the distribution behavior of main arithmetic operators in mathematics. Although the original AOA has shown well competitive performance with popular meta-heuristic algorithms, it still faces the issues of insufficient exploitation ability, ease of falling into local optima and low convergence accuracy in large-scale applications. In order to ameliorate these deficiencies, an enhanced hybrid AOA named CSOAOA, integrated with point set strategy, optimal neighborhood learning strategy and crisscross strategy, is developed in this paper. First, a good point set initialization strategy is added to obtain a higher-quality initial population, which improves the convergence speed of the algorithm. Then, the optimal neighborhood learning strategy is adopted to guide the individual's search behavior and avoid the algorithm falling into the current local optimum, which boosts the search efficiency and calculation accuracy. Finally, by combining AOA with the crisscross optimization algorithm, the exploration and utilization ability of the crisscross algorithm are integrated into the CSOAOA. These strategies collaborate to enhance AOA in accelerating overall performance. The superiority of the proposed CSOAOA is comprehensively verified by comparing with the original AOA, six improved AOA and numerous celebrated and newly developed algorithms on the well-known 23 classical benchmark functions, IEEE Congress on Evolutionary Computation (CEC) 2019 test suite and IEEE CEC 2020 benchmark functions, respectively. Meanwhile, the practicability of CSOAOA is also highlighted by solving eight real-world engineering design problems. Furthermore, the statistical testing of CSOAOA has been conducted to validate its significance. Experimental results and statistical comparisons manifest the superior performance of CSOAOA over the comparison algorithms in terms of precision, convergence rate and solution quality. Therefore, CSOAOA is potentially a powerful and competitive meta-heuristic algorithm for solving complex engineering optimization problems.

mechanics,engineering, multidisciplinary,mathematics, interdisciplinary applications
A compact multiagent system based on autonomy oriented computing

Xiao-Feng Xie,Jiming Liu

DOI: https://doi.org/10.1109/IAT.2005.6

2005-01-01

Abstract:A compact multiagent optimization system (MAOSC) based on autonomy oriented computing (AOC) is presented. Performed by a society of autonomous entities in iterative cycles, an optimization algorithm can simply be described by a macro generate-and-test behavior, which deploys a few elemental generating behaviors under conditioned reflex behaviors supported by a testing operation library. MAOSC provides a simple framework for not only realizing and comparing algorithms, but also deploying evolvable algorithms. The experimental results of MAOSC cases on benchmark functions are compared with those of other algorithms, which show its efficiency.
A self-stabilizing and auto-provisioning orchestration for microservices in edge-cloud continuum

Binlei Cai,Xiaoli Wang,Bin Wang,Meihong Yang,Ying Guo,Qin Guo

DOI: https://doi.org/10.1016/j.comnet.2024.110279

IF: 5.493

2024-02-23

Computer Networks

Abstract:Modern user-facing services are progressively evolving from large monolithic applications to complex graphs of loosely-coupled microservices. While this shift provides opportunities to offload some microservices of a user-facing service to edge devices that are close to the end users, it also complicates the application deployment and resource provisioning in the edge-cloud continuum, due to complex relationships across microservices and unstable public networks. To reduce resource wastage and improve user experience, this paper presents SMO, a self-managed orchestration system for microservices. SMO leverages a self-stabilizing placement mechanism to optimally deploy microservices through perceiving both interferences and communication overheads. During runtime, it further tailors a multi-agent deep deterministic policy gradient (MADDPG)-based model in combination with attention and prioritized replay, which automatically provisions appropriate resources for each microservice subject to the differentiated tail latency Service Level Objectives (SLOs) of multiple user workloads. Experimental results demonstrate that SMO saves up to 39% of CPU cores and 47% of memory footprint while providing guarantees for heterogeneous tail latency SLOs.

computer science, information systems,telecommunications,engineering, electrical & electronic, hardware & architecture
A Scenario-Oriented Benchmark for Assessing AIOps Algorithms in Microservice Management

Yongqian Sun,Jiaju Wang,Zhengdan Li,Xiaohui Nie,Minghua Ma,Shenglin Zhang,Yuhe Ji,Lu Zhang,Wen Long,Hengmao Chen,Yongnan Luo,Dan Pei

2024-07-09

Abstract:AIOps algorithms play a crucial role in the maintenance of microservice systems. Many previous benchmarks' performance leaderboard provides valuable guidance for selecting appropriate algorithms. However, existing AIOps benchmarks mainly utilize offline datasets to evaluate algorithms. They cannot consistently evaluate the performance of algorithms using real-time datasets, and the operation scenarios for evaluation are static, which is insufficient for effective algorithm selection. To address these issues, we propose an evaluation-consistent and scenario-oriented evaluation framework named MicroServo. The core idea is to build a live microservice benchmark to generate real-time datasets and consistently simulate the specific operation scenarios on it. MicroServo supports different leaderboards by selecting specific algorithms and datasets according to the operation scenarios. It also supports the deployment of various types of algorithms, enabling algorithms hot-plugging. At last, we test MicroServo with three typical microservice operation scenarios to demonstrate its efficiency and usability.

Distributed, Parallel, and Cluster Computing,Machine Learning
AOAAO: The Hybrid Algorithm of Arithmetic Optimization Algorithm With Aquila Optimizer

Yu-Jun Zhang,Yu-Xin Yan,Juan Zhao,Zheng-Ming Gao

DOI: https://doi.org/10.1109/access.2022.3144431

IF: 3.9

2022-01-01

IEEE Access

Abstract:Many new algorithms have been proposed to solve the mathematical equations formulated to describe the real-world problems. But there still does not exist one algorithm that could solve the problems all. And most of the proposed algorithms have defects in some aspects, they need to be improved in application. In order to find a more efficient optimization algorithm and inspired by the better performance of the Arithmetic Optimization algorithm (AOA) and Aquila Optimizer (AO), we proposed a hybridization algorithm of them and abbreviated AOAAO in this paper. Considering the better performance of the Harris Hawk optimization (HHO) algorithm, an energy parameter E was also introduced to balance the exploration and exploitation procedures of individuals in AOAAO swarms, and furthermore, piecewise linear map was introduced to decrease the randomness of the energy parameter. Pseudo code of the proposed AOAAO algorithm was presented, Simulation experiments were carried out on the benchmark functions and three classical engineering problems were also involved in optimization. Nine popular well demonstrated algorithms were included for comparison. Results confirmed the AOAAO would be more efficient in optimization with faster convergence rate, and higher convergence accuracy.

computer science, information systems,telecommunications,engineering, electrical & electronic
Prometheus and AIOps for the orchestration of Cloud-native applications in Ananke

Alessandro Di Stefano,Antonella Di Stefano,Giovanni Morana,Daniele Zito

DOI: https://doi.org/10.1109/wetice53228.2021.00017

2021-10-01

Abstract:DevOps methodologies reduce the gap between Developers and Operations teams, enabling automation, integration, monitoring, and team collaboration by exploiting continuous integration and deployment and providing high-scale performance for the cloud software delivery life-cycle. AIOps seems to represent the future of IT automation by leveraging DevOps methodologies, Big Data, and AI-enabled strategies for the smart orchestration of Cloud-native applications. Based on the Ananke monitoring model, this paper investigates design issues and strategies required to enable integration between clusters and applications managed by Ananke, and their metrics stored in the Prometheus Monitoring system. Some classic algorithms for anomaly detection and forecasting of time series have been introduced in the proposed AIOps Prometheus Framework for the system analysis and orchestration of applications. Finally, we propose, as a case study, an auto-scaling strategy based on the prediction of traffic peeks for a web application using the Facebook Prophet model.
Multi-objective ant lion optimizer: a multi-objective optimization algorithm for solving engineering problems

Seyedali Mirjalili,Pradeep Jangir,Shahrzad Saremi

DOI: https://doi.org/10.1007/s10489-016-0825-8

IF: 5.3

2016-07-25

Applied Intelligence

Abstract:This paper proposes a multi-objective version of the recently proposed Ant Lion Optimizer (ALO) called Multi-Objective Ant Lion Optimizer (MOALO). A repository is first employed to store non-dominated Pareto optimal solutions obtained so far. Solutions are then chosen from this repository using a roulette wheel mechanism based on the coverage of solutions as antlions to guide ants towards promising regions of multi-objective search spaces. To prove the effectiveness of the algorithm proposed, a set of standard unconstrained and constrained test functions is employed. Also, the algorithm is applied to a variety of multi-objective engineering design problems: cantilever beam design, brushless dc wheel motor design, disk brake design, 4-bar truss design, safety isolating transformer design, speed reduced design, and welded beam deign. The results are verified by comparing MOALO against NSGA-II and MOPSO. The results of the proposed algorithm on the test functions show that this algorithm benefits from high convergence and coverage. The results of the algorithm on the engineering design problems demonstrate its applicability is solving challenging real-world problems as well.

computer science, artificial intelligence
Boosting Aquila Optimizer by Marine Predators Algorithm for Combinatorial Optimization

Shuang Wang,Heming Jia,Abdelazim G Hussien,Laith Abualigah,Guanjun Lin,Hongwei Wei,Zhenheng Lin,Krishna Gopal Dhal

DOI: https://doi.org/10.1093/jcde/qwae004

2024-01-19

Journal of Computational Design and Engineering

Abstract:In this study, an improved version of Aquila Optimizer (AO) known as EHAOMPA has been developed by using the Marine Predators Algorithm (MPA). MPA is a recent and well-behaved optimizer with a unique memory saving and FADs mechanism. At the same time, it suffers from various defects such as inadequate global search, sluggish convergence, and stagnation of local optima. However, AO has contented robust global exploration capability, fast convergence speed, and high search efficiency. Thus, the proposed EHAOMPA aims to complement the shortcomings of AO and MPA while bringing new features. Specifically, the representative-based hunting technique is incorporated into the exploration stage to enhance population diversity. At the same time, random opposition-based learning (ROBL) is introduced into the exploitation stage to prevent the optimizer from sticking to local optima. This study tests the performance of EHAOMPA's on twenty-three standard mathematical benchmark functions, 29 complex test functions from the CEC2017 test suite, six constrained industrial engineering design problems, and a CNN-hyperparameter optimization for COVID-19 CT-image detection problem. EHAOMPA is compared with four existing optimization algorithm types, achieving the best performance on both numerical and practical issues. Compared to other methods, the test function results demonstrate that EHAOMPA exhibits a more potent global search capability, a higher convergence rate, increased accuracy, and an improved ability to avoid local optima. The excellent experimental results in practical problems indicate that the developed EHAOMPA has great potential in solving real-world optimization problems. The combination of multiple strategies can effectively improve the performance of the algorithm. The source code of the EHAOMPA is publicly available at https://github.com/WangShuang92/EHAOMPA.

computer science, interdisciplinary applications,engineering, multidisciplinary
OML: Open, Monetizable, and Loyal AI

Zerui Cheng,Edoardo Contente,Ben Finch,Oleg Golev,Jonathan Hayase,Andrew Miller,Niusha Moshrefi,Anshul Nasery,Sandeep Nailwal,Sewoong Oh,Himanshu Tyagi,Pramod Viswanath

2024-11-02

Abstract:Artificial Intelligence (AI) has steadily improved across a wide range of tasks. However, the development and deployment of AI are almost entirely controlled by a few powerful organizations that are racing to create Artificial General Intelligence (AGI). The centralized entities make decisions with little public oversight, shaping the future of humanity, often with unforeseen consequences. In this paper, we propose OML, which stands for Open, Monetizable, and Loyal AI, an approach designed to democratize AI development. OML is realized through an interdisciplinary framework spanning AI, blockchain, and cryptography. We present several ideas for constructing OML using technologies such as Trusted Execution Environments (TEE), traditional cryptographic primitives like fully homomorphic encryption and functional encryption, obfuscation, and AI-native solutions rooted in the sample complexity and intrinsic hardness of AI tasks. A key innovation of our work is introducing a new scientific field: AI-native cryptography. Unlike conventional cryptography, which focuses on discrete data and binary security guarantees, AI-native cryptography exploits the continuous nature of AI data representations and their low-dimensional manifolds, focusing on improving approximate performance. One core idea is to transform AI attack methods, such as data poisoning, into security tools. This novel approach serves as a foundation for OML 1.0 which uses model fingerprinting to protect the integrity and ownership of AI models. The spirit of OML is to establish a decentralized, open, and transparent platform for AI development, enabling the community to contribute, monetize, and take ownership of AI models. By decentralizing control and ensuring transparency through blockchain technology, OML prevents the concentration of power and provides accountability in AI development that has not been possible before.

Artificial Intelligence,Cryptography and Security
OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents

Qiang Sun,Yuanyi Luo,Sirui Li,Wenxiao Zhang,Wei Liu

2024-08-06

Abstract:Multimodal conversational agents are highly desirable because they offer natural and human-like interaction. However, there is a lack of comprehensive end-to-end solutions to support collaborative development and benchmarking. While proprietary systems like GPT-4o and Gemini demonstrating impressive integration of audio, video, and text with response times of 200-250ms, challenges remain in balancing latency, accuracy, cost, and data privacy. To better understand and quantify these issues, we developed OpenOmni, an open-source, end-to-end pipeline benchmarking tool that integrates advanced technologies such as Speech-to-Text, Emotion Detection, Retrieval Augmented Generation, Large Language Models, along with the ability to integrate customized models. OpenOmni supports local and cloud deployment, ensuring data privacy and supporting latency and accuracy benchmarking. This flexible framework allows researchers to customize the pipeline, focusing on real bottlenecks and facilitating rapid proof-of-concept development. OpenOmni can significantly enhance applications like indoor assistance for visually impaired individuals, advancing human-computer interaction. Our demonstration video is available <a class="link-external link-https" href="https://www.youtube.com/watch?v=zaSiT3clWqY" rel="external noopener nofollow">this https URL</a>, demo is available via <a class="link-external link-https" href="https://openomni.ai4wa.com" rel="external noopener nofollow">this https URL</a>, code is available via <a class="link-external link-https" href="https://github.com/AI4WA/OpenOmniFramework" rel="external noopener nofollow">this https URL</a>.

Human-Computer Interaction,Artificial Intelligence,Computation and Language
Optimizing Cloud Performance: A Microservice Scheduling Strategy for Enhanced Fault-Tolerance, Reduced Network Traffic, and Lower Latency

Abdullah Alelyani,Amitava Datta,Ghulam Mubashar Hassan

DOI: https://doi.org/10.1109/access.2024.3373316

IF: 3.9

2024-03-12

IEEE Access

Abstract:The emergence of microservice architecture has brought significant advancements in software development, offering improved scalability and availability of applications. Cloud computing benefits from microservice architecture by mitigating the risks of single failures and ensuring compliance with service-level agreements. However, using microservice architecture presents two challenges: 1) managing network traffic, which leads to latency and network congestion; and 2) inefficient resource allocation for microservices. Current approaches have limitations in addressing these challenges. To overcome these limitations, we propose a novel scheduling strategy that schedules microservice replicas using a modified particle swarm optimization algorithm to place them on the most suitable physical machine. Additionally, we balance the load across physical machines in the cluster using a simple round-robin algorithm. Furthermore, our scheduling strategy integrates with Kubernetes to tackle resource allocation and deployment challenges. The proposed strategy has been evaluated by simulating two scenarios using Alibaba and Google datasets. The experimental results demonstrate the effectiveness of our strategy in reducing traffic, balancing load, and utilizing CPU and memory efficiently.

computer science, information systems,telecommunications,engineering, electrical & electronic
Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

Weize Chen,Jiarui Yuan,Chen Qian,Cheng Yang,Zhiyuan Liu,Maosong Sun

2024-10-11

Abstract:Large Language Model (LLM) based multi-agent systems (MAS) show remarkable potential in collaborative problem-solving, yet they still face critical challenges: low communication efficiency, poor scalability, and a lack of effective parameter-updating optimization methods. We present Optima, a novel framework that addresses these issues by significantly enhancing both communication efficiency and task effectiveness in LLM-based MAS through LLM training. Optima employs an iterative generate, rank, select, and train paradigm with a reward function balancing task performance, token efficiency, and communication readability. We explore various RL algorithms, including Supervised Fine-Tuning, Direct Preference Optimization, and their hybrid approaches, providing insights into their effectiveness-efficiency trade-offs. We integrate Monte Carlo Tree Search-inspired techniques for DPO data generation, treating conversation turns as tree nodes to explore diverse interaction paths. Evaluated on common multi-agent tasks, including information-asymmetric question answering and complex reasoning, Optima shows consistent and substantial improvements over single-agent baselines and vanilla MAS based on Llama 3 8B, achieving up to 2.8x performance gain with less than 10\% tokens on tasks requiring heavy information exchange. Moreover, Optima's efficiency gains open new possibilities for leveraging inference-compute more effectively, leading to improved inference-time scaling laws. By addressing fundamental challenges in LLM-based MAS, Optima shows the potential towards scalable, efficient, and effective MAS (<a class="link-external link-https" href="https://chenweize1998.github.io/optima-project-page" rel="external noopener nofollow">this https URL</a>).

Computation and Language,Artificial Intelligence
LLM Agent Operating System

Kai Mei,Zelong Li,Shuyuan Xu,Ruosong Ye,Yingqiang Ge,Yongfeng Zhang

DOI: https://doi.org/10.48550/arxiv.2403.16971

2024-03-25

Operating Systems

Abstract:The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS). Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future. The project is open-source at https://github.com/agiresearch/AIOS.
AIOS: LLM Agent Operating System

Kai Mei,Zelong Li,Shuyuan Xu,Ruosong Ye,Yingqiang Ge,Yongfeng Zhang

2024-03-26

Abstract:The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS) as the brain of the OS, enabling an operating system "with soul" -- an important step towards AGI. Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future. The project is open-source at

Operating Systems,Artificial Intelligence,Computation and Language

AOMO - An AI-aided Optimizer for Microservices Orchestration.

Service-oriented open architecture for complex process system modeling and optimization

Towards Automated Inter-Service Authorization for Microservice Applications.

How autonomy oriented computing (AOC) tackles a computationally hard optimization problem

Many-objective ant lion optimizer (MaOALO): A new many-objective optimizer with its engineering applications

Machine Learning Insides OptVerse AI Solver: Design Principles and Applications

An improved ant colony optimization algorithm for QoS-aware dynamic web service composition

An enhanced hybrid arithmetic optimization algorithm for engineering applications

A compact multiagent system based on autonomy oriented computing

A self-stabilizing and auto-provisioning orchestration for microservices in edge-cloud continuum

A Scenario-Oriented Benchmark for Assessing AIOps Algorithms in Microservice Management

AOAAO: The Hybrid Algorithm of Arithmetic Optimization Algorithm With Aquila Optimizer

Prometheus and AIOps for the orchestration of Cloud-native applications in Ananke

Multi-objective ant lion optimizer: a multi-objective optimization algorithm for solving engineering problems

Boosting Aquila Optimizer by Marine Predators Algorithm for Combinatorial Optimization

OML: Open, Monetizable, and Loyal AI

OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents

Optimizing Cloud Performance: A Microservice Scheduling Strategy for Enhanced Fault-Tolerance, Reduced Network Traffic, and Lower Latency

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

LLM Agent Operating System

AIOS: LLM Agent Operating System