Abstract:The increasing expansion of Internet-of-Things (IoT) in the world requires Big Data analytic infrastructures to produce valuable knowledge in IoT applications. IoT includes devices with limited resources, whereby it requires efficient platforms to process massive data obtained from sensors. Nowadays, many IoT applications such as audio and video recognition depend on state-of-the-art Deep Neural Networks (DNNs). Therefore, we need to execute DNNs on IoT devices. DNNs offer excellent recognition accuracy but they suffer from high computational and memory resource demands. Due to these constraints, currently, IoT applications that depend on deep learning are mostly offloaded to cloudlets and clouds. Offloading imposes extra network bandwidth consumption costs in addition to delayed response for IoT devices. In this paper, we propose a method that instead of using all layers of DNN for inference, only selects a subset of layers that provide sufficient accuracy for each task. We propose AdaInNet, a method to significantly reduce computational cost and network latency in DNN-based IoT applications while maintaining prediction accuracy based on Distributed DNNs (DDNNs). The method uses modified Distributed DNNs with early exits in order to minimize computation costs and network latency by selecting sub-layers or exit branches of DDNNs with early exits. We also proposed a hybrid Classifier-Wise (CW)—Interactive learning method for the training of DDNNs and Agent's networks. Furthermore, we create a custom agent model for the Advantage Actor-Critic Deep Reinforcement Learning method in order to preserve recognition accuracy while utilizing a minimum number of layers. Finally, we execute the extensive numerical simulation, in order to evaluate and compare our proposed AdaInNet method with rival methods under standard CIFAR 100 and CIFAR 10 datasets and ResNet-110 and ResNet-32 DNNs which are used in IoT applications in previous works. The results provide strong quantitative evidence that the AdaInNet method not only accelerates inference but also reduces computational cost and latency.

Multi-Fine-Grained DNNs Partition and Offloading over Fog Computing Networks

Mobility-Aware Task Offloading and Migration Schemes in Fog Computing Networks.

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

DNNOff: Offloading DNN-Based Intelligent IoT Applications in Mobile Edge Computing

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

Energy-Efficient AI over a Virtualized Cloud Fog Network

Accelerating Deep Learning Inference via Model Parallelism and Partial Computation Offloading

DDPQN: An Efficient DNN Offloading Strategy in Local-Edge-Cloud Collaborative Environments

Adaptive Partitioning and Efficient Scheduling for Distributed DNN Training in Heterogeneous IoT Environment

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

Multiagent DDPG-Based Joint Task Partitioning and Power Control in Fog Computing Networks

Joint DNN Partition and Resource Allocation for Task Offloading in Edge-Cloud-Assisted IoT Environments

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning

AdaInNet: an adaptive inference engine for distributed deep neural networks offloading in IoT-FOG applications based on reinforcement learning

Distributed Deep Neural Networks with System Cost Minimization in Fog Networks

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

Guardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud

EosDNN: An Efficient Offloading Scheme for DNN Inference Acceleration in Local-Edge-Cloud Collaborative Environments

Resource-aware Deployment of Dynamic DNNs over Multi-tiered Interconnected Systems

End-Edge Collaborative Inference of Convolutional Fuzzy Neural Networks for Big Data-Driven Internet of Things

An Adaptive DNN Inference Acceleration Framework with End–edge–cloud Collaborative Computing