On-device Training: A First Overview on Existing Systems

Shuai Zhu,Thiemo Voigt,JeongGil Ko,Fatemeh Rahimian

DOI: https://doi.org/10.1145/3696003

2024-09-23

Abstract:The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (ii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work targets to summarize and analyze state-of-the-art systems research that allows such on-device model training capabilities and provide a survey of on-device training from a systems perspective.

Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper "On-device Training: A First Overview on Existing Systems" aims to address the challenges of training machine learning (ML) and deep learning (DL) models on resource-constrained devices. Specifically, the paper focuses on the following aspects:

1. **Privacy Protection**: By training models on local devices, it avoids uploading data to the cloud, thereby protecting user data privacy.
2. **Personalization and Environmental Adaptation**: Local training allows devices to adjust models based on specific user usage or environmental changes, improving model accuracy and applicability.
3. **Applications in Remote and Low Connectivity Scenarios**: In the absence of stable internet connections, local training ensures model updates and optimizations, suitable for applications in remote areas such as healthcare and environmental monitoring.

### Main Challenges

The paper points out the main technical challenges in achieving on-device training:

1. **Hardware Resource Mismatch**: IoT devices typically have limited hardware resources, such as memory capacity ranging from a few KB to a few MB, while model training requires substantial computational and memory resources.
2. **High Heterogeneity**: IoT devices vary widely, from microcontrollers to single-board computers, with significant differences in capabilities, making it difficult to propose a universal solution.
3. **Limitations of Existing Work**: Most existing model training optimization algorithms focus on improving accuracy or accelerating convergence, neglecting the resource constraints of edge devices, such as memory and battery life.

### Research Scope

The paper summarizes and analyzes the current state-of-the-art on-device training systems, covering various resource-constrained devices from modern mobile platforms to microcontrollers. Additionally, the paper discusses system design on smartphones but excludes the NVIDIA Jetson platform. The paper primarily focuses on single-device training, excluding collaborative model training (e.g., federated learning).

### Summary

By reviewing existing systems, the paper aims to provide researchers and developers with a comprehensive perspective on the current state and future directions of on-device training, thereby promoting further research and application in this field.