Abstract:Large models like transformer models cannot be directly applied and trained at the network edge since edge devices are heterogeneous and often equipped with limited resources. To collaboratively train a large model at the network edge, existing works assign sub-models of the large model with proper sizes to each edge device, and adopt advanced technologies like knowledge distillation to aggregate their local models to update the large model. However, these methods assign sub-models in a coarse-grained manner and require a certain amount of edge devices to train the large model locally, thus cannot achieve efficient edge learning for large models in practice. In this paper, to make the utmost of edge devices to collaboratively train a large model faster and better, we propose a novel synchronous edge learning framework to achieve efficient large model training across heterogeneous resource-limited edge devices. Specifically, to reduce the waiting time in the training process while ensuring that every edge device can afford the local training, we design a capability-aware local model customization mechanism to granularly tailor personalized model structures for each edge device based on their memory and computing capabilities to make them have similar local training times. Then, to efficiently integrate the local training achievements of heterogeneous edge devices, we propose a layer augmentation-based heterogeneous model aggregation mechanism to align local models and quickly and effectively get the global update for the global large model. Besides, we propose a monument distillation-based model deployment mechanism to deploy the updated global large model at edge devices without the loss of learned knowledge. In this way, each edge device can fully contribute to the global large model training, thus achieving efficient edge learning for the large model. Experimental results demonstrate that our framework has better performance in accuracy and efficiency than the state-of-art edge learning frameworks in the heterogeneous environment.

EASTER: Learning to Split Transformers at the Edge Robustly

ED-ViT: Splitting Vision Transformer for Distributed Inference on Edge Devices

Bridging the Resource Gap: Deploying Advanced Imitation Learning Models onto Affordable Embedded Platforms

Optimizing the Deployment of Tiny Transformers on Low-Power MCUs

Towards Efficient Edge Learning for Large Models in Heterogeneous Resource-limited Environments.

TinyFormer: Efficient Transformer Design and Deployment on Tiny Devices

EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms

Adaptive Offloading of Transformer Inference for Weak Edge Devices with Masked Autoencoders

DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices

ProTransformer: Robustify Transformers via Plug-and-Play Paradigm

Efficient Deployment of Transformer Models in Analog In-Memory Computing Hardware

ALBERTA: ALgorithm-Based Error Resilience in Transformer Architectures

Vision Transformer Computation and Resilience for Dynamic Inference

Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment

Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization

On the Performance and Memory Footprint of Distributed Training: An Empirical Study on Transformers

Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search

Distributed Inference with Minimal Off-Chip Traffic for Transformers on Low-Power MCUs

Reinforced Transformer Learning for VSI-DDoS Detection in Edge Clouds

MCUFormer: Deploying Vision Tranformers on Microcontrollers with Limited Memory.

Skipformer: Evolving Beyond Blocks for Extensively Searching On-Device Language Models With Learnable Attention Window