Abstract:The ongoing realization of a transition towards increased reliance on edge computing is evident. This shift not only underscores the importance of interdependence in knowledge sharing but also contributes to a reduction in adaptation and transmission time through cloud computing directly, along with decreased computational demands compared to edge-to-edge computing. A persistent challenge in knowledge transfer stems from the shift in dependence from high-performance computing systems to compact models, making the adjustment cumbersome. Even with cross-edge knowledge transfer, pre-processing remains essential to facilitate seamless adaptation—a necessity that endures despite the escalating volume of data. Consequently, the system tends to become overburdened with instance filtering instead. To address this issue, we present an optimization for small-to-small inter-edge transfer learning of a comparable size. This is achieved through a self-supervised domain-adaptive protocol, utilizing self-target curriculum-based pseudo-labeling creation, new statistical alignment, and lightweight attention. In the multi-view verification using public visual DA datasets, we demonstrate a remarkable improvement of 73.6 (+4.2 %) in Top-1 performance. Our results consistently outperform other methods, particularly when using the latent feature input technique. This advantage is most noticeable at the 10-th layer for latent feature input. Crucially, our approach incurs the modest increase in computational demands, adding just 100 K-units of parameters and MFLOPS of computational power. Furthermore, through model enhancement via acceleration and compression, we achieve significant compression—up to 6.28 times faster at FP32 and 7.18 times faster at FP16 runtimes—resulting in an increased FPS to 124.9 and 142.9, respectively. This demonstration effectively illustrates the feasibility of advancing inter-edge transfer learning with our approach.

Advances in inter-edge transfer learning with self-curriculum-labeling adaptive learning and lightweight attention

Edge-cloud Collaborative Learning with Federated and Centralized Features

Importance-Aware Data Selection and Resource Allocation in Federated Edge Learning System.

AccEPT: an Acceleration Scheme for Speeding Up Edge Pipeline-parallel Training

Enabling Deep Learning on Edge Devices through Filter Pruning and Knowledge Transfer

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Effective Domain Knowledge Transfer with Soft Fine-tuning

Exploring the Use of Synthetic Gradients for Distributed Deep Learning Across Cloud and Edge Resources.

Energy-Efficient Edge Learning via Joint Data Deepening-and-Prefetching

MobileDA: Toward Edge-Domain Adaptation

Transferring Knowledge from Large Foundation Models to Small Downstream Models

Knowledge Distillation with Attention for Deep Transfer Learning of Convolutional Networks

Adaptive knowledge transfer for class incremental learning

Adaptive Weighting via Federated Evaluation Mechanism for Domain Adaptation with Edge Devices

Towards Compute-Optimal Transfer Learning

Maintaining Symmetry between Convolutional Neural Network Accuracy and Performance on an Edge TPU with a Focus on Transfer Learning Adjustments

Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation

Center-Sensitive Kernel Optimization for Efficient On-Device Incremental Learning

EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Voting

Memory-efficient Deep Learning Inference with Incremental Weight Loading and Data Layout Reorganization on Edge Systems.

Efficient Continual Learning with Low Memory Footprint For Edge Device