Abstract:Large Language Models (LLMs) have demonstrated impressive capabilities in autogenerating code based on natural language instructions provided by humans. We observed that in the microservice models of edge computing, the problem of deployment latency optimization can be transformed into an NP-hard mathematical optimization problem. However, in the real world, deployment strategies at the edge often require immediate updates, while human-engineered code tends to be lagging. To bridge this gap, we innovatively integrated LLMs into the decision-making process for microservice deployment. Initially, we constructed a private Retrieval Augmented Generation (RAG) database containing prior knowledge. Subsequently, we employed meticulously designed step-by-step inductive instructions and used the chain of thought (CoT) technique to enable the LLM to learn, reason, reflect, and regenerate. We decomposed the microservice deployment latency optimization problem into a collection of granular sub-problems (described in natural language), progressively providing instructions to the fine-tuned LLM to generate corresponding code blocks. The generated code blocks underwent integration and consistency assessment. Additionally, we prompted the LLM to generate code without the use of the RAG database for comparative analysis. We executed the aforementioned code and comparison algorithm under identical operational environments and simulation parameters, conducting rigorous result analysis. Our fine-tuned model significantly reduced latencies by 22.8% in handling surges in request flows, 37.8% in managing complex microservice types, and 39.5% in processing increased network nodes compared to traditional algorithms. Moreover, our approach demonstrated marked improvements in latency performance over LLMs not utilizing RAG technology and reinforcement learning algorithms reported in other literature. The use of LLMs also highlights the concept of symmetry, as the symmetrical structure of input-output relationships in microservice deployment models aligns with the LLM's inherent ability to process and generate balanced and optimized code. Symmetry in this context allows for more efficient resource allocation and reduces redundant operations, further enhancing the model's effectiveness. We believe that LLMs hold substantial potential in optimizing microservice deployment models.

Software Service Engineering in the Era of Large Language Models

Towards an Understanding of Large Language Models in Software Engineering Tasks

A Survey on Large Language Models for Software Engineering

Large Language Models for Software Engineering: Survey and Open Problems

Large Language Models for Software Engineering: A Systematic Literature Review

A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends

A Software Engineering Perspective on Testing Large Language Models: Research, Practice, Tools and Benchmarks

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

A systematic study in intelligent software engineering based on Large Language Model

Software Testing with Large Language Models: Survey, Landscape, and Vision

Impact of Large Language Models on Generating Software Specifications

Large Language Models (LLMs): Deployment, Tokenomics and Sustainability

S3LLM: Large-Scale Scientific Software Understanding with LLMs using Source, Metadata, and Document

Large Language Model-Based Agents for Software Engineering: A Survey

On the use of Large Language Models in Model-Driven Engineering

LLM4DS: Evaluating Large Language Models for Data Science Code Generation

LLMs are Imperfect, Then What? An Empirical Study on LLM Failures in Software Engineering

Optimizing Microservice Deployment in Edge Computing with Large Language Models: Integrating Retrieval Augmented Generation and Chain of Thought Techniques

Requirements are All You Need: From Requirements to Code with LLMs

Software Development in the Age of Intelligence: Embracing Large Language Models with the Right Approach