Abstract:Edge intelligence in space-air-ground integrated networks (SAGINs) can enable worldwide network coverage beyond geographical limitations for users to access ubiquitous and low-latency intelligence services. Facing global coverage and complex environments in SAGINs, edge intelligence can provision approximate large language models (LLMs) agents for users via edge servers at ground base stations (BSs) or cloud data centers relayed by satellites. As LLMs with billions of parameters are pre-trained on vast datasets, LLM agents have few-shot learning capabilities, e.g., chain-of-thought (CoT) prompting for complex tasks, which raises a new trade-off between resource consumption and performance in SAGINs. In this paper, we propose a joint caching and inference framework for edge intelligence to provision sustainable and ubiquitous LLM agents in SAGINs. We introduce "cached model-as-a-resource" for offering LLMs with limited context windows and propose a novel optimization framework, i.e., joint model caching and inference, to utilize cached model resources for provisioning LLM agent services along with communication, computing, and storage resources. We design "age of thought" (AoT) considering the CoT prompting of LLMs, and propose a least AoT cached model replacement algorithm for optimizing the provisioning cost. We propose a deep Q-network-based modified second-bid (DQMSB) auction to incentivize network operators, which can enhance allocation efficiency by 23% while guaranteeing strategy-proofness and free from adverse selection.

What problem does this paper attempt to address?

The paper aims to address the issue of providing large language model (LLMs) proxy services with edge intelligence in Space-Air-Ground Integrated Networks (SAGINs). Specifically, the paper focuses on the following aspects: 1. **Resource Consumption and Performance Trade-off**: In SAGINs, due to the wide geographical coverage and complex environment, edge intelligence needs to provide approximate large-scale language model (LLMs) proxy services to users through ground base stations (BSs) or cloud data centers relayed by satellites. These LLMs have a large number of parameters and are pre-trained on large-scale datasets, capable of few-shot learning, such as Chain-of-Thought (CoT) prompting to handle complex tasks. However, this brings new trade-off issues between resource consumption and performance. 2. **Joint Caching and Inference Framework**: To provide sustainable and ubiquitous LLMs proxy services in SAGINs, the authors propose a joint caching and inference framework. This framework introduces "caching models as a resource" into edge intelligence, similar to traditional communication, computing, and storage resources, and designs a novel optimization framework—joint model caching and inference—to utilize these cached model resources. 3. **Age of Thought (AoT) Metric**: Considering the chain-of-thought capability of LLMs, the paper designs a specific metric—AoT, to measure the relevance and coherence of intermediate thoughts during CoT inference. Based on this metric, a minimum AoT cache replacement algorithm is proposed to determine which cached models should be removed, thereby reducing the service provision cost for operators. 4. **Improved Auction Mechanism**: To address the adverse selection problem caused by information asymmetry when satellites act as simple relays, the paper combines a modified second-price auction (Modified Second-Bid, MSB) and deep reinforcement learning (DRL) to design a deep Q-network-based MSB auction (DQMSB). This mechanism selects the optimal price adjustment factor to improve the efficiency of network operator allocation while ensuring strategy-proofness and immunity to adverse selection. In summary, the main contribution of this paper is the proposal of a novel optimization framework to provide sustainable and extensive LLMs proxy services to users through satellites and ground base stations in SAGINs, along with the design of corresponding caching algorithms and market mechanisms to address this issue.

Cached Model-as-a-Resource: Provisioning Large Language Model Agents for Edge Intelligence in Space-air-ground Integrated Networks

Large Language Models (llms) Inference Offloading and Resource Allocation in Cloud-Edge Networks: an Active Inference Approach

Joint Foundation Model Caching and Inference of Generative AI Services for Edge Intelligence

Reinforcement learning for cost-effective IoT service caching at the edge

Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks

Joint Model Caching and Resource Allocation in Generative AI-Enabled Wireless Edge Networks

Two-Timescale Model Caching and Resource Allocation for Edge-Enabled AI-Generated Content Services

When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment

Collaborative Computation Offloading and Resource Management in Space–Air–Ground Integrated Networking: A Deep Reinforcement Learning Approach

Edge-LLM: A Collaborative Framework for Large Language Model Serving in Edge Computing

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization

Optimizing Microservice Deployment in Edge Computing with Large Language Models: Integrating Retrieval Augmented Generation and Chain of Thought Techniques

On Optimal Caching and Model Multiplexing for Large Model Inference

An Online Approach for DNN Model Caching and Processor Allocation in Edge Computing

Joint Offloading and Resource Allocation for Hybrid Cloud and Edge Computing in SAGINs: A Decision Assisted Hybrid Action Space Deep Reinforcement Learning Approach

Harnessing Your DRAM and SSD for Sustainable and Accessible LLM Inference with Mixed-Precision and Multi-level Caching

Joint Coded Caching and Resource Allocation for Multimedia Service in Space-Air-Ground Integrated Networks

Hybrid SLM and LLM for Edge-Cloud Collaborative Inference

AoI-Aware Optimization of Service Caching-Assisted Offloading and Resource Allocation in Edge Cellular Networks