Abstract:Deep Learning approaches based on Convolutional Neural Networks (CNNs) are extensively utilized and very successful in a wide range of application areas, including image classification and speech recognition. For the execution of trained CNNs, i.e. model inference, we nowadays witness a shift from the Cloud to the Edge. Unfortunately, deploying and inferring large, compute and memory intensive CNNs on edge devices is challenging because these devices typically have limited power budgets and compute/memory resources. One approach to address this challenge is to leverage all available resources across multiple edge devices to deploy and execute a large CNN by properly partitioning the CNN and running each CNN partition on a separate edge device. Although such distribution, deployment, and execution of large CNNs on multiple edge devices is a desirable and beneficial approach, there currently does not exist a design and programming framework that takes a trained CNN model, together with a CNN partitioning specification, and fully automates the CNN model splitting and deployment on multiple edge devices to facilitate distributed CNN inference at the Edge. Therefore, in this paper, we propose a novel framework, called AutoDiCE, for automated splitting of a CNN model into a set of sub-models and automated code generation for distributed and collaborative execution of these sub-models on multiple, possibly heterogeneous, edge devices, while supporting the exploitation of parallelism among and within the edge devices. Our experimental results show that AutoDiCE can deliver distributed CNN inference with reduced energy consumption and memory usage per edge device, and improved overall system throughput at the same time.

Edge Intelligence Based Co-training of CNN.

Researching the CNN Collaborative Inference Mechanism for Heterogeneous Edge Devices

Extendable Multi-Device Collaborative Pipeline Parallel Inference in the Edge-Cloud Scenario

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Cooperative Training Video Surveillance Technology under the Edge Computing

AccEPT: an Acceleration Scheme for Speeding Up Edge Pipeline-parallel Training

EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

Cooperative Inference with Interleaved Operator Partitioning for CNNs

DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices

Collaborative edge computing for distributed CNN inference acceleration using receptive field-based segmentation

Automated Exploration and Implementation of Distributed CNN Inference at the Edge

Importance-Aware Data Selection and Resource Allocation in Federated Edge Learning System.

Distributed Convolutional Neural Network Training on Mobile and Edge Clusters

DynaComm: Accelerating Distributed CNN Training between Edges and Clouds through Dynamic Communication Scheduling

HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing

Attention-based Feature Compression for CNN Inference Offloading in Edge Computing

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

AutoDiCE: Fully Automated Distributed CNN Inference at the Edge

Towards Efficient Compact Network Training on Edge-Devices

DeeperThings: Fully Distributed CNN Inference on Resource-Constrained Edge Devices