Abstract:The increasing expansion of Internet-of-Things (IoT) in the world requires Big Data analytic infrastructures to produce valuable knowledge in IoT applications. IoT includes devices with limited resources, whereby it requires efficient platforms to process massive data obtained from sensors. Nowadays, many IoT applications such as audio and video recognition depend on state-of-the-art Deep Neural Networks (DNNs). Therefore, we need to execute DNNs on IoT devices. DNNs offer excellent recognition accuracy but they suffer from high computational and memory resource demands. Due to these constraints, currently, IoT applications that depend on deep learning are mostly offloaded to cloudlets and clouds. Offloading imposes extra network bandwidth consumption costs in addition to delayed response for IoT devices. In this paper, we propose a method that instead of using all layers of DNN for inference, only selects a subset of layers that provide sufficient accuracy for each task. We propose AdaInNet, a method to significantly reduce computational cost and network latency in DNN-based IoT applications while maintaining prediction accuracy based on Distributed DNNs (DDNNs). The method uses modified Distributed DNNs with early exits in order to minimize computation costs and network latency by selecting sub-layers or exit branches of DDNNs with early exits. We also proposed a hybrid Classifier-Wise (CW)—Interactive learning method for the training of DDNNs and Agent's networks. Furthermore, we create a custom agent model for the Advantage Actor-Critic Deep Reinforcement Learning method in order to preserve recognition accuracy while utilizing a minimum number of layers. Finally, we execute the extensive numerical simulation, in order to evaluate and compare our proposed AdaInNet method with rival methods under standard CIFAR 100 and CIFAR 10 datasets and ResNet-110 and ResNet-32 DNNs which are used in IoT applications in previous works. The results provide strong quantitative evidence that the AdaInNet method not only accelerates inference but also reduces computational cost and latency.

DNNOff: Offloading DNN-Based Intelligent IoT Applications in Mobile Edge Computing

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Pre-DNNOff: On-Demand DNN Model Offloading Method for Mobile Edge Computing

Distributed Assignment With Load Balancing for DNN Inference at the Edge

DDPQN: An Efficient DNN Offloading Strategy in Local-Edge-Cloud Collaborative Environments

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

An Intelligent Dynamic Offloading From Cloud to Edge for Smart IoT Systems With Big Data

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

AdaInNet: an adaptive inference engine for distributed deep neural networks offloading in IoT-FOG applications based on reinforcement learning

Dependent Task Scheduling Using Parallel Deep Neural Networks in Mobile Edge Computing

Accelerating Deep Learning Inference via Model Parallelism and Partial Computation Offloading

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

Joint Architecture Design and Workload Partitioning for DNN Inference on Industrial IoT Clusters

Task Partitioning and Offloading in DNN-Task Enabled Mobile Edge Computing Networks

EosDNN: An Efficient Offloading Scheme for DNN Inference Acceleration in Local-Edge-Cloud Collaborative Environments

Dynamic DNN Decomposition for Lossless Synergistic Inference

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

EdgeKE: An On-Demand Deep Learning IoT System for Cognitive Big Data on Industrial Edge Devices

Deep Learning on Mobile Devices Through Neural Processing Units and Edge Computing

Intelligent Online Computation Offloading for Wireless-Powered Mobile-Edge Computing