Abstract:With the rise of artificial intelligence (AIoT), integrating deep neural networks (DNNs) with mobile and embedded devices has become an increasingly growing trend. This integration enhances the ability of mobile IoT devices to collect and analyze data. The previous integration paradigm mainly relied on cloud based training and terminal deployment. However, the real world is dynamic and changing, and previous integration paradigms can lead to untimely model updates, decreased accuracy, and significant communication overhead. Therefore, in order to ensure the accuracy of DNN, the on device training method has gradually become a research hotspot. However, the bottleneck that restricts the efficiency of on device training is usually limited local perception data and computing resources. To address this bottleneck, researchers have proposed federated learning. However, in order to protect data privacy, federated learning cannot share data or model details, which brings new problems such as slow model convergence and reduced convergence accuracy. By contrast, we consider that the environment in which mobile devices operate in the real world is trustworthy (such as personal devices in smart spaces, trusted devices from the same organization/company, etc.). This trustworthiness promotes the sharing of data and model training tasks among mobile devices to improve training efficiency and model accuracy. Therefore, in this article, we propose CrowdLearning, a decentralized distributed training framework based on a collection of trusted AIoT devices. It consists of two collaborative modules, namely the heterogeneous resource sensitive task offloading module that solves the training delay bottleneck and the communication efficient data reallocation module that determines when, how, and to whom data is transmitted, improving the efficiency and effectiveness of DNN training. The experimental results show that group learning outperforms existing federated learning and distributed training baselines on devices in different scenarios. It achieved a 55.8% reduction in training latency and a 67.1% reduction in communication costs.

MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing

Close the Gap Between Deep Learning and Mobile Intelligence by Incorporating Training in the Loop

Collaborative Learning Between Cloud and End Devices

Explore Training of Deep Convolutional Neural Networks on Battery-powered Mobile Devices: Design and Application

Deep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions

Understanding Sensor Data Using Deep Learning Methods on Resource-Constrained Edge Devices.

Distributed Learning on Mobile Devices: A New Approach to Data Mining in the Internet of Things

Semi-Supervised Decentralized Machine Learning with Device-to-Device Cooperation

A Unified Federated DNNs Framework for Heterogeneous Mobile Devices

Energy-Efficient Federated Training on Mobile Device

Privacy‐preserving task offloading in mobile edge computing: A deep reinforcement learning approach

Adaptive Dataset Management Scheme for Lightweight Federated Learning in Mobile Edge Computing

Mobile Device Training Strategies in Federated Learning: an Evolutionary Game Approach

Enhancing Security-Problem-Based Deep Learning in Mobile Edge Computing

Zone-based Federated Learning for Mobile Sensing Data

Cloud-Edge-Client Collaborative Learning in Digital Twin Empowered Mobile Networks

Smart Collaborative Optimizations Strategy for Mobile Edge Computing Based on Deep Reinforcement Learning

CrowdLearning: A Decentralized Distributed Training Framework Based on Collectives of Trusted AIoT Devices

DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning

Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness

On-Device Learning for Model Personalization with Large-Scale Cloud-Coordinated Domain Adaption