Abstract:Modern consumer electronic devices have started executing deep learning-based intelligence services on devices, not cloud servers, to keep personal data on devices and to reduce network and cloud costs. We find such a trend as the opportunity to personalize intelligence services by updating neural networks with user data without exposing the data out of devices: on-device training. However, the limited resources of devices incurs significant difficulties. We propose a light-weight on-device training framework, NNTrainer, which provides highly memory-efficient neural network training techniques and proactive swapping based on fine-grained execution order analysis for neural networks. Moreover, its optimizations do not sacrifice accuracy and are transparent to training algorithms; thus, prior algorithmic studies may be implemented on top of NNTrainer. The evaluations show that NNTrainer can reduce memory consumption down to 1/20 (saving 95%!) and effectively personalizes intelligence services on devices. NNTrainer is cross-platform and practical open-source software, which is being deployed to millions of mobile devices.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to address the challenges of personalized training (on - device training) of deep - learning models on consumer electronic devices. Specifically, the authors focus on the following key issues: 1. **Protecting user privacy**: The traditional cloud - based training method requires uploading users' personal data to the cloud, which may lead to privacy leakage. By training directly on the device, personal data can be ensured not to leave the device, thus better protecting user privacy. 2. **Reducing network and cloud costs**: With the increase in the number of smart devices, transmitting all devices' data to the cloud for processing will result in huge network bandwidth and cloud - computing resource consumption. Training on the device can significantly reduce these costs. 3. **Improving the degree of service personalization**: General pre - trained models may not be well - adapted to each user's specific needs. By using users' local data to fine - tune or update the model, the accuracy of the model and the user experience can be improved. 4. **Coping with the problem of limited device resources**: Consumer electronic devices (such as smart phones, smart TVs, etc.) usually have limited computing and memory resources, which makes it very difficult to directly train deep - learning models on the device. To this end, the authors propose a lightweight training framework NNTrainer to optimize resource utilization and ensure efficient on - device training without sacrificing accuracy and latency. ### Main contributions of NNTrainer - **Efficient memory management**: Through fine - grained execution order analysis and proactive swapping technology, NNTrainer can significantly reduce memory consumption, enabling complex neural network models to be trained on resource - constrained embedded devices. - **Cross - platform support**: NNTrainer is an open - source, cross - platform framework that has been deployed on millions of mobile devices and is applicable to multiple operating systems (such as Ubuntu, Tizen, Android, and Windows). - **Maintaining accuracy**: The optimization methods of NNTrainer do not sacrifice the accuracy of the model and can be transparently applied to existing training algorithms, allowing previous research results to be directly utilized. Through these improvements, NNTrainer can not only effectively support on - device AI training and personalized services, but also provide new ideas and technical means for future edge - computing and privacy - protection research.

A New Frontier of AI: On-Device AI Training and Personalization

Explore Training of Deep Convolutional Neural Networks on Battery-powered Mobile Devices: Design and Application

Close the Gap Between Deep Learning and Mobile Intelligence by Incorporating Training in the Loop

On-Device Training Under 256KB Memory

TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

An Overview of Energy-Efficient Hardware Accelerators for On-Device Deep-Neural-Network Training

On-device Training: A First Overview on Existing Systems

How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures

NNStreamer: Efficient and Agile Development of On-Device AI Systems

Automated Customization of On-Device Inference for Quality-of-Experience Enhancement

CAMEL: Co-Designing AI Models and Embedded DRAMs for Efficient On-Device Learning

Swan: A Neural Engine for Efficient DNN Training on Smartphone SoCs

3U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration

Toward Among-Device AI from On-Device AI with Stream Pipelines

A Survey on the Optimization of Neural Network Accelerators for Micro-AI On-Device Inference

Towards Implementing Energy-aware Data-driven Intelligence for Smart Health Applications on Mobile Platforms

NNStreamer: Stream Processing Paradigm for Neural Networks, Toward Efficient Development and Execution of On-Device AI Applications

On-Device Neural Net Inference with Mobile GPUs

Low-Energy On-Device Personalization for MCUs