Abstract:Recent advances in deep neural networks (DNNs) have greatly improved the accuracy and universality of various intelligent applications, at the expense of increasing model size and computational demand. Since the resources of end devices are often too limited to deploy a complete DNN model, offloading DNN inference tasks to cloud servers is a common approach to meet this gap. However, due to the limited bandwidth of WAN and the long distance between end devices and cloud servers, this approach may lead to significant data transmission latency. Therefore, device-edge collaborative inference has emerged as a promising paradigm to accelerate the execution of DNN inference tasks where DNN models are partitioned to be sequentially executed in both end devices and edge servers. Nevertheless, collaborative inference in heterogeneous edge environments with multiple edge servers, end devices and DNN tasks has been overlooked in previous research. To fill this gap, we investigate the optimization problem of collaborative inference in a heterogeneous system and propose a scheme CIS, i.e., collaborative inference scheme, which jointly combines DNN partition, task offloading and scheduling to reduce the average weighted inference latency. CIS decomposes the problem into three parts to achieve the optimal average weighted inference latency. In addition, we build a prototype that implements CIS and conducts extensive experiments to demonstrate the scheme's effectiveness and efficiency. Experiments show that CIS reduces 29% to 71% on the average weighted inference latency compared to the other four existing schemes.

Poster: AIDE: An Accurate Internet Distance Estimation System

Replacing Network Coordinate System with Internet Delay Matrix Service (IDMS): A Case Study in Chinese Internet

Phoenix: Towards an Accurate, Practical and Decentralized Network Coordinate System

Two-layer Network Coordinate System for Internet Distance Prediction.

WiEdge: Edge Computing for Audio Sensing Applications with Accurate Wireless Link Prediction.

Taming the triangle inequality violations with network coordinate system on real internet.

Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization

DISCS: A Distributed Coordinate System Based on Robust Nonnegative Matrix Completion

Efficient Channel Estimation for RIS-Aided MIMO Communications with Unitary Approximate Message Passing

On the stability of network distance estimation.

NCShield: Securing Decentralized, Matrix Factorization-Based Network Coordinate Systems

WDCS: A weight-based distributed coordinate system

Pharos: A Decentralized And Hierarchical Network Coordinate System For Internet Distance Prediction

Differentiated Content Acquisition Integrating Network Coding and Edge Computing for Industrial Internet

Network on chip for enterprise information management and integration in intelligent physical systems

Handling Triangle Inequality Violations in Euclidean Distance Based Network Coordinate Systems

Collaborative Inference for Deep Neural Networks in Edge Environments

A Data-Driven and the Deep Learning Based CDN Recommendation Framework for ICPs.

DID: Distributed Incremental Block Coordinate Descent for Nonnegative Matrix Factorization

Pndp: Predicting Accurate Network Distance Based On Clustering

Mutual Information Evolution Based Performance Analysis in IDMA System