Abstract:In recent years, deep neural networks (DNNs) have witnessed a booming of artificial intelligence Internet of Things applications with stringent demands across high accuracy and low latency. A widely adopted solution is to process such computation-intensive DNNs inference tasks with edge computing. Nevertheless, existing edge-based DNN processing methods still cannot achieve acceptable performance due to the intensive transmission data and unnecessary computation. To address the above limitations, we take the advantage of Multi-exit DNNs (ME-DNNs) that allows the tasks to exit early at different depths of the DNN during inference, based on the input complexity. However, naively deploying ME-DNNs in edge still fails to deliver fast and consistent inference in the wild environment. Specifically, 1) at the model-level, unsuitable exit settings will increase additional computational overhead and will lead to excessive queuing delay; 2) at the computation-level, it is hard to sustain high performance consistently in the dynamic edge computing environment. In this paper, we present a Low Latency Edge Intelligence Scheme based on Multi-Exit DNNs (LEIME) to tackle the aforementioned problem. At the model-level, we propose an exit setting algorithm to automatically build optimal ME-DNNs with lower time complexity; At the computation-level, we present a distributed offloading mechanism to fine-tune the task dispatching at runtime to sustain high performance in the dynamic environment, which has the property of close-to-optimal performance guarantee. Finally, we implement a prototype system and extensively evaluate it through testbed and large-scale simulation experiments. Experimental results demonstrate that LEIME significantly improves applications' performance, achieving 1.1–18.7 × speedup in different situations.

Poster Abstract: E4: Energy-Efficient Early-Exit DNN Inference Framework for Edge Video Analytics

Poster Abstract: E4: Energy-Efficient Early-Exit DNN Inference Framework for Edge Video Analytics.

AyE-Edge: Automated Deployment Space Search Empowering Accuracy Yet Efficient Real-Time Object Detection on the Edge

Elastic DNN Inference with Unpredictable Exit in Edge Computing

A Feedback-Driven DNN Inference Acceleration System for Edge-Assisted Video Analytics

Enabling Low Latency Edge Intelligence Based on Multi-exit DNNs in the Wild

Poster Abstract: DVFO: Dynamic Voltage, Frequency and Offloading for Efficient AI on Edge Devices

EdgeEye: A Data-Driven Approach for Optimal Deployment of Edge Video Analytics

EdgeNN: Efficient Neural Network Inference for CPU-GPU Integrated Edge Devices.

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

Energy-Aware Workload Allocation for Distributed Deep Neural Networks in Edge-Cloud Continuum.

DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference

TLEE: Temporal-wise and Layer-wise Early Exiting Network for Efficient Video Recognition on Edge Devices

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

Energy-efficient Content-aware DNN Inference for Mobile Video Via Deep Reinforcement Learning

Sustainable Edge Intelligence Through Energy-Aware Early Exiting

Dynamic DNN Model Selection and Inference off Loading for Video Analytics with Edge-Cloud Collaboration

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

EdgeKE: An On-Demand Deep Learning IoT System for Cognitive Big Data on Industrial Edge Devices

DNN Inference Acceleration with Partitioning and Early Exiting in Edge Computing

Toward Efficient and Adaptive Design of Video Detection System with Deep Neural Networks