Abstract:As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy. Edgent exploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in real-world deployment, Edgent is properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly, Edgent derives the best configurations with the assist of regression-based prediction models, while in a dynamic environment where the bandwidth varies dramatically, Edgent generates the best execution plan through the online change point detection algorithm that maps the current bandwidth state to the optimal configuration. We implement Edgent prototype based on the Raspberry Pi and the desktop PC and the extensive experimental evaluations demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices.

Accelerating DNN Inference by Edge-Cloud Collaboration

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

AccEPT: an Acceleration Scheme for Speeding Up Edge Pipeline-parallel Training

DNN Inference Acceleration with Partitioning and Early Exiting in Edge Computing

Deep Neural Network Task Partitioning and Offloading for Mobile Edge Computing

ADDA: Adaptive Distributed DNN Inference Acceleration in Edge Computing Environment

Communication-Efficient Separable Neural Network for Distributed Inference on Edge Devices

DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

Joint multi-user DNN partitioning and task offloading in mobile edge computing

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

A DNN inference acceleration algorithm combining model partition and task allocation in heterogeneous edge computing system

A Novel Adaptive Computation Offloading Strategy for Collaborative DNN Inference over Edge Devices.

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

Joint Optimization of Data Transfer and Co-Execution for DNN in Edge Computing

Energy-Efficient Joint Partitioning and Offloading for Delay-Sensitive CNN Inference in Edge Computing

Partitioning and Deployment of Deep Neural Networks on Edge Clusters

Distributed Assignment With Load Balancing for DNN Inference at the Edge