Abstract:In some applications, edge learning is experiencing a shift in focusing from conventional learning from scratch to new two-stage learning unifying pre-training and task-specific fine-tuning. This paper considers the problem of joint communication and computation resource management in a two-stage edge learning system. In this system, model pre-training is first conducted at an edge server via centralized learning on local pre-stored general data, and then task-specific fine-tuning is performed at edge devices based on the pre-trained model via federated edge learning. For the two-stage learning model, we first analyze the convergence behavior (in terms of the average squared gradient norm bound), which characterizes the impacts of various system parameters such as the number of learning rounds and batch sizes in the two stages on the convergence rate. Based on our analytical results, we then propose a joint communication and computation resource management design to minimize an average squared gradient norm bound, subject to constraints on the transmit power, overall system energy consumption, and training delay. The decision variables include the number of learning rounds, batch sizes, clock frequencies, and transmit power control for both pre-training and fine-tuning stages. Finally, numerical results are provided to evaluate the effectiveness of our proposed design. It is shown that the proposed joint resource management over the pre-training and fine-tuning stages well balances the system performance trade-off among the training accuracy, delay, and energy consumption. The proposed design is also shown to effectively leverage the inherent trade-off between pre-training and fine-tuning, which arises from the differences in data distribution between pre-stored general data versus real-time task-specific data, thus efficiently optimizing overall system performance.

Exploring Edge-driven Collaborative Fine-tuning Towards Customized AIGC Services

Edge AIBench: Towards Comprehensive End-to-End Edge Computing Benchmarking.

Edge-cloud Collaborative Learning with Federated and Centralized Features

Offloading and Quality Control for AI Generated Content Services in 6G Mobile Edge Computing Networks

Towards Integrated Fine-tuning and Inference when Generative AI meets Edge Intelligence

Implementation of Big AI Models for Wireless Networks with Collaborative Edge Computing

Joint Foundation Model Caching and Inference of Generative AI Services for Edge Intelligence

Rethinking Resource Management in Edge Learning: A Joint Pre-training and Fine-tuning Design Paradigm

An Edge-Cloud Collaboration Framework for Generative AI Service Provision with Synergetic Big Cloud Model and Small Edge Models

Joint Model Assignment and Resource Allocation for Cost-Effective Mobile Generative Services

Cloud-Edge-Terminal Collaborative AIGC for Autonomous Driving

Sparks of GPTs in Edge Intelligence for Metaverse: Caching and Inference for Mobile AIGC Services

Towards Multi-Task Generative-AI Edge Services with an Attention-based Diffusion DRL Approach

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

Multi-Agent RL-Based Industrial AIGC Service Offloading over Wireless Edge Networks

In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning

Optimizing AIGC Services by Prompt Engineering and Edge Computing: A Generative Diffusion Model-Based Contract Theory Approach

Online Fine-tuning Method for Power Grid Artificial Intelligence Model Based on Cloud-Edge Collaboration

Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services

An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learning