Generative AI as a Service in 6G Edge-Cloud: Generation Task Offloading by In-context Learning

Hao Zhou,Chengming Hu,Dun Yuan,Ye Yuan,Di Wu,Xue Liu,Zhu Han,Charlie Zhang

2024-08-05

Abstract:Generative artificial intelligence (GAI) is a promising technique towards 6G networks, and generative foundation models such as large language models (LLMs) have attracted considerable interest from academia and telecom industry. This work considers a novel edge-cloud deployment of foundation models in 6G networks. Specifically, it aims to minimize the service delay of foundation models by radio resource allocation and task offloading, i.e., offloading diverse content generation tasks to proper LLMs at the network edge or cloud. In particular, we first introduce the communication system model, i.e., allocating radio resources and calculating link capacity to support generated content transmission, and then we present the LLM inference model to calculate the delay of content generation. After that, we propose a novel in-context learning method to optimize the task offloading decisions. It utilizes LLM's inference capabilities, and avoids the difficulty of dedicated model training or fine-tuning as in conventional machine learning algorithms. Finally, the simulations demonstrate that the proposed edge-cloud deployment and in-context learning task offloading method can achieve satisfactory generation service quality without dedicated model training or fine-tuning.

Systems and Control

What problem does this paper attempt to address?

The paper aims to address the task offloading problem of Generative Artificial Intelligence (GAI) in 6G networks, with specific objectives including:

1. **Minimizing Service Delay**: Reducing the service delay of foundational models (such as large language models, LLMs) through wireless resource allocation and task offloading strategies.
2. **Flexible Task Handling**: Designing a novel edge-cloud collaborative deployment strategy, allowing small LLMs to handle low-latency tasks at the network edge, while large LLMs handle tasks with high-quality requirements in the cloud.
3. **Proposing New Decision-Making Methods**: Introducing an in-context learning-based approach to optimize task offloading decisions, avoiding the complex model training and fine-tuning processes of traditional machine learning algorithms.

Through the above research, the paper demonstrates that the proposed edge-cloud deployment architecture and in-context learning method can achieve satisfactory generative service quality without the need for specialized training or fine-tuning.