Abstract:By combining edge computing and parallel computing, distributed edge computing has emerged as a new paradigm to exploit the booming IoT devices at the edge. To accelerate computation at the edge, i.e. , the inference tasks for DNN-driven applications, the parallelism of both computation and communication needs to be considered for distributed edge computing, and thus, the problem of Minimum Latency joint Communication and Computation Scheduling (MLCCS) is proposed. However, existing works have rigid assumptions that the communication time of each device is fixed and the workload can be split arbitrarily small. Aiming at making the work more practical and general, the MLCCS problem without the above assumptions is studied in this paper. Firstly, the MLCCS problem under a general model is formulated and proved to be NP-hard. Secondly, a pyramid-based computing model is proposed to consider the parallelism of communication and computation jointly, which has an approximation ratio of $1+\delta$ , where $\delta$ is related to devices' communication rates. An interesting property under such a computing model is identified and proved, i.e. , the optimal latency can be obtained under arbitrary scheduling order when all the devices share the same communication rate. When the workload cannot be split arbitrarily, an approximation algorithm with a ratio of at most $2\cdot (1+\delta )$ is proposed. Additionally, for handling the dynamically changing network scenarios, several algorithms are also proposed accordingly. Finally, the theoretical analysis and simulation results verify that the proposed algorithm has high performance in terms of latency. Two testbed experiments are also conducted, which show that the proposed method outperforms the existing methods, reducing the latency by up to 29.2% for inference tasks at the edge.

An open source framework based on Kafka-ML for Distributed DNN inference over the Cloud-to-Things continuum

Managing and Deploying Distributed and Deep Neural Models Through Kafka-ML in the Cloud-to-Things Continuum

Review of Inference Time Prediction Approaches of DNN: Emphasis on Service Robots with Cloud-Edge-device Architecture

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Extendable Multi-Device Collaborative Pipeline Parallel Inference in the Edge-Cloud Scenario

DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters

Towards Real-Time Inference Offloading with Distributed Edge Computing: the Framework and Algorithms

A collaborative cloud-edge computing framework in distributed neural network

The orchestration of Machine Learning frameworks with data streams and GPU acceleration in Kafka‐ML: A deep‐learning performance comparative

Distributed Deep Neural Networks over the Cloud, the Edge and End Devices

Toward Collaborative Inferencing of Deep Neural Networks on Internet-of-Things Devices

Functions as a service for distributed deep neural network inference over the cloud‐to‐things continuum

An Adaptive DNN Inference Acceleration Framework with End–edge–cloud Collaborative Computing

Joint Architecture Design and Workload Partitioning for DNN Inference on Industrial IoT Clusters

Edge-Cloud Cooperation for DNN Inference Via Reinforcement Learning and Supervised Learning

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

DEFER: Distributed Edge Inference for Deep Neural Networks

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

T2C: A Multi-User System for Deploying DNNs in a Thing-to-Cloud Continuum

Inference Time Optimization Using BranchyNet Partitioning

Automated Exploration and Implementation of Distributed CNN Inference at the Edge