Abstract:With the rapid advancement of artificial intelligence, generative artificial intelligence (GAI) has taken a leading role in transforming data processing methods. However, the high computational demands of GAI present challenges for devices with limited resources. As we move towards the sixth generation of mobile networks (6G), the higher data rates and improved energy efficiency of 6G create a need for more efficient data processing in GAI. Traditional GAI, however, shows its limitations in meeting these demands. To address these challenges, we introduce a multi-level collaborative inference system designed for next-generation networks and communication. Our proposed system features a deployment strategy that assigns models of varying sizes to devices at different network layers. Then, we design a task offloading strategy to optimise both efficiency and latency. Furthermore, a modified early exit mechanism is implemented to enhance the inference process for single models. Experimental results demonstrate that our system effectively reduces inference latency while maintaining high-quality output. Specifically, compared to existing work, our system can reduce inference time by up to 17% without sacrificing the inference accuracy.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the challenges of efficient deployment and inference of generative artificial intelligence (GAI) in the sixth - generation mobile network (6G) environment. Specifically, the paper focuses on the following key issues: 1. **High demand for computing resources**: - GAI models usually require a large amount of computing resources, which makes them difficult to run on resource - constrained devices (such as smart phones or embedded systems). - The deployment of these models in practical applications is limited by computing resources, affecting their feasibility in real - world scenarios. 2. **Communication latency issues**: - Although handing complex inference tasks to the cloud can take advantage of the powerful computing power of the cloud platform, it will introduce new problems, especially communication latency. - Data transmission latency and model calculation latency together constitute inference latency, which may be unacceptable in time - sensitive applications. 3. **Limitations of existing solutions**: - Although the existing multi - model collaborative inference schemes reduce the calculation latency and cost, they lack fine - grained model selection and may lead to accuracy loss. - Other research works on accelerating inference also fail to fully consider the impact of communication latency in mobile networks and communication environments. To solve these problems, the paper proposes a multi - level collaborative inference system, aiming to optimize the data processing efficiency in next - generation networks and communications. The contributions of this system include: - **Distributed multi - level inference framework**: Applicable to next - generation networks and communications, it can maintain low overall latency while ensuring inference quality. - **Task offloading strategy**: Optimize the efficiency and latency of the deployed model system through a special confidence algorithm and pruning technology based on the attention mechanism. - **Enhanced early - exit mechanism**: Optimized for single - model inference, improving system performance and reducing inference latency. - **Experimental verification**: Experiments show the effectiveness of the system. Compared with existing works, this system can reduce the inference time by up to 17% without sacrificing inference accuracy. ### Summary The main goal of the paper is to solve the problems of efficient deployment and inference of GAI models in the 6G environment by designing a multi - level collaborative inference system, thereby achieving low - latency, high - quality inference services.

Distributed Collaborative Inference System in Next-Generation Networks and Communication

Collaborative Inference for Deep Neural Networks in Edge Environments

GainNet: Coordinates the Odd Couple of Generative AI and 6G Networks

Towards Integrated Fine-tuning and Inference when Generative AI meets Edge Intelligence

Task-Oriented 6G Native-AI Network Architecture

Generative AI-Driven Digital Twin for Mobile Networks

GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning

Collaborative Inference for Large Models with Task Offloading and Early Exiting

Communication-Efficient Edge AI Inference Over Wireless Networks

User-Centric Interactive AI for Distributed Diffusion Model-based AI-Generated Content

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

Distributed Foundation Models for Multi-Modal Learning in 6G Wireless Networks

DGI: Easy and Efficient Inference for GNNs

Exploring Collaborative Distributed Diffusion-Based AI-Generated Content (AIGC) in Wireless Networks

A Fine-Grained End-to-End Latency Optimization Framework for Wireless Collaborative Inference

Interactive AI with Retrieval-Augmented Generation for Next Generation Networking

Communication-Efficient Separable Neural Network for Distributed Inference on Edge Devices

Distributed Intelligence in Wireless Networks

DISTRIBUTED ARTIFICIAL INTELLIGENCE MODELS IN PERSPECTIVE COMMUNICATION NETWORKS

Collaborative Inference Acceleration Integrating DNN Partitioning and Task Offloading in Mobile Edge Computing