Abstract:By combining edge computing and parallel computing, distributed edge computing has emerged as a new paradigm to exploit the booming IoT devices at the edge. To accelerate computation at the edge, i.e. , the inference tasks for DNN-driven applications, the parallelism of both computation and communication needs to be considered for distributed edge computing, and thus, the problem of Minimum Latency joint Communication and Computation Scheduling (MLCCS) is proposed. However, existing works have rigid assumptions that the communication time of each device is fixed and the workload can be split arbitrarily small. Aiming at making the work more practical and general, the MLCCS problem without the above assumptions is studied in this paper. Firstly, the MLCCS problem under a general model is formulated and proved to be NP-hard. Secondly, a pyramid-based computing model is proposed to consider the parallelism of communication and computation jointly, which has an approximation ratio of $1+\delta$ , where $\delta$ is related to devices' communication rates. An interesting property under such a computing model is identified and proved, i.e. , the optimal latency can be obtained under arbitrary scheduling order when all the devices share the same communication rate. When the workload cannot be split arbitrarily, an approximation algorithm with a ratio of at most $2\cdot (1+\delta )$ is proposed. Additionally, for handling the dynamically changing network scenarios, several algorithms are also proposed accordingly. Finally, the theoretical analysis and simulation results verify that the proposed algorithm has high performance in terms of latency. Two testbed experiments are also conducted, which show that the proposed method outperforms the existing methods, reducing the latency by up to 29.2% for inference tasks at the edge.

Cascade: A Platform for Delay-Sensitive Edge Intelligence

CascadeServe: Unlocking Model Cascades for Inference Serving

Cascade: An Application Pipelining Toolkit for Coarse-Grained Reconfigurable Arrays

MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference

CASCADE: A Framework for CNN Accelerator Synthesis with Concatenation and Refreshing Dataflow

IDK Cascades: Fast Deep Learning by Learning not to Overthink

Towards Real-Time Inference Offloading with Distributed Edge Computing: the Framework and Algorithms

MultiTASC: A Multi-Tenancy-Aware Scheduler for Cascaded DNN Inference at the Consumer Edge

Hetero-Edge: Orchestration of Real-time Vision Applications on Heterogeneous Edge Clouds

At-the-edge Data Processing for Low Latency High Throughput Machine Learning Algorithms

Research on Convolutional Neural Network Inference Acceleration and Performance Optimization for Edge Intelligence

Game-Based Adaptive FLOPs and Partition Point Decision Mechanism With Latency and Energy-Efficient Tradeoff for Edge Intelligence

Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving

HE2C: A Holistic Approach for Allocating Latency-Sensitive AI Tasks across Edge-Cloud

CascadeHD: Efficient Many-Class Learning Framework Using Hyperdimensional Computing

AI on the Edge: Rethinking AI-based IoT Applications Using Specialized Edge Architectures

Ultra-Reliable and Low-Latency Computing in the Edge with Kubernetes

GOURD: Tensorizing Streaming Applications to Generate Multi-Instance Compute Platforms

Benchmarking Edge AI Platforms for High-Performance ML Inference

Decomposable Intelligence on Cloud-Edge IoT Framework for Live Video Analytics

Elastic high-performance computing platform for real-time data analysis