Abstract:Deep learning (DL) applications have attracted significant attention with the rapidly growing demand for Internet of Things (IoT) systems. However, performing the inference tasks for DL applications on IoT devices is challenging due to the large computational demands of DL models. Recently, edge computing has offered us a solution by deploying resources near the end users. However, resources at the edge are still limited; thus, management issues, such as allocating the networking resources as well as the computing capabilities and configuring the devices appropriately for different applications, become essential. For knobs in such edge management, we consider multiple application tasks with different options of DL models and different hyperparameter settings, along with possible decomposition points that utilize the split DL concept to design the configuration tables. Layer-level decomposition in split DL provides greater flexibility by splitting a single DL inference model into parts on different computing devices, and each part consists of several consecutive layers. We then propose the SplitDL-Image and the SplitDL-Video algorithms based on the Vickrey–Clarke–Groves (VCG) mechanism by considering model performance and frames per second (FPS) requirements with the preferences of the heterogeneous IoT devices. The proposed method allocates networking and edge server computing resources according to the designed configuration tables by assigning the appropriate configuration to each IoT device. Simulation results based on real-world applications show that the proposed method indeed allocates more resources to IoT devices with more urgent/important tasks, preference for better accuracy, or higher local computational cost. In addition, other desired properties, such as truthful bidding, individual rationality, and weakly budget balance, are also guaranteed.

Decomposable Intelligence on Cloud-Edge IoT Framework for Live Video Analytics

Towards Memory-Efficient Inference in Edge Video Analytics

Edge–IoT Computing and Networking Resource Allocation for Decomposable Deep Learning Inference

HCOME: Research on Hybrid Computation Offloading Strategy for MEC Based on DDPG

Intelligent Video Surveillance Based on Mobile Edge Networks

Streaming Video Analytics On The Edge With Asynchronous Cloud Support

OsmoticGate: Adaptive Edge-Based Real-Time Video Analytics for the Internet of Things

IDEC: Intelligent Distributed Edge Computing System Architecture Enabling Deep Learning across Heterogeneous IoT Devices

Decentralized Edge Intelligence-Driven Network Resource Orchestration Mechanism

AI on the Edge: Rethinking AI-based IoT Applications Using Specialized Edge Architectures

Multi-stream Adaptive Offloading of Joint Compressed Video Streams, Feature Streams, and Semantic Streams in Edge Computing Systems.

Computational Offloading in Semantic-Aware Cloud-Edge-End Collaborative Networks

Joint Multi-User DNN Partitioning and Computational Resource Allocation for Collaborative Edge Intelligence

Edge Computing Enabled Real-Time Video Analysis via Adaptive Spatial-Temporal Semantic Filtering

DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

Online Learning for Orchestration of Inference in Multi-User End-Edge-Cloud Networks

ACE: Towards Application-Centric Edge-Cloud Collaborative Intelligence

EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters

Task-Oriented Communication for Edge Video Analytics