Abstract:Large Language Models (LLMs) have demonstrated impressive capabilities in autogenerating code based on natural language instructions provided by humans. We observed that in the microservice models of edge computing, the problem of deployment latency optimization can be transformed into an NP-hard mathematical optimization problem. However, in the real world, deployment strategies at the edge often require immediate updates, while human-engineered code tends to be lagging. To bridge this gap, we innovatively integrated LLMs into the decision-making process for microservice deployment. Initially, we constructed a private Retrieval Augmented Generation (RAG) database containing prior knowledge. Subsequently, we employed meticulously designed step-by-step inductive instructions and used the chain of thought (CoT) technique to enable the LLM to learn, reason, reflect, and regenerate. We decomposed the microservice deployment latency optimization problem into a collection of granular sub-problems (described in natural language), progressively providing instructions to the fine-tuned LLM to generate corresponding code blocks. The generated code blocks underwent integration and consistency assessment. Additionally, we prompted the LLM to generate code without the use of the RAG database for comparative analysis. We executed the aforementioned code and comparison algorithm under identical operational environments and simulation parameters, conducting rigorous result analysis. Our fine-tuned model significantly reduced latencies by 22.8% in handling surges in request flows, 37.8% in managing complex microservice types, and 39.5% in processing increased network nodes compared to traditional algorithms. Moreover, our approach demonstrated marked improvements in latency performance over LLMs not utilizing RAG technology and reinforcement learning algorithms reported in other literature. The use of LLMs also highlights the concept of symmetry, as the symmetrical structure of input-output relationships in microservice deployment models aligns with the LLM's inherent ability to process and generate balanced and optimized code. Symmetry in this context allows for more efficient resource allocation and reduces redundant operations, further enhancing the model's effectiveness. We believe that LLMs hold substantial potential in optimizing microservice deployment models.

Retrieval-Augmented Generation for Mobile Edge Computing via Large Language Model

Latency-Aware Resource Allocation for Mobile Edge Generation and Computing via Deep Reinforcement Learning

Joint Optimization Of Computation Offloading And Ul/Dl Resource Allocation In Mec Systems

Latency Optimization for Resource Allocation in Mobile-Edge Computation Offloading.

Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures

Dynamic Offloading Strategy for Delay-Sensitive Task in Mobile-Edge Computing Networks

Energy and Delay-Aware Task Offloading and Resource Allocation in Mobile Edge Computing

Generative Inference of Large Language Models in Edge Computing: An Energy Efficient Approach

Revenue-Maximized Offloading Decision And Fine-Grained Resource Allocation In Edge Network

An Efficient Online Computation Offloading Approach for Large-Scale Mobile Edge Computing via Deep Reinforcement Learning

Joint Optimization of Offloading and Communication Resources in Mobile Edge Computing

Energy-Saving Offloading by Jointly Allocating Radio and Computational Resources for Mobile Edge Computing

Joint Offloading and Resource Allocation Using Deep Reinforcement Learning in Mobile Edge Computing

Resource Allocation for Stable LLM Training in Mobile Edge Computing

BiLSTM-based Federated Learning Computation Offloading and Resource Allocation Algorithm in MEC

Resource allocation algorithm for MEC based on Deep Reinforcement Learning

Optimizing Microservice Deployment in Edge Computing with Large Language Models: Integrating Retrieval Augmented Generation and Chain of Thought Techniques

Ultra-Low Latency Multi-Task Offloading in Mobile Edge Computing

Multi-Agent Reinforcement Learning For Joint Wireless And Computational Resource Allocation In Mobile Edge Computing System

A Method for Mobile-Aware Multi-User Offloading in Hybrid Cloud-Edge Environment

Towards Decentralized Task Offloading and Resource Allocation in User-Centric Mobile Edge Computing