Abstract:With the development of deep learning technologies and edge computing, the combination of them can make artificial intelligence ubiquitous. Due to the constrained computation resources of the edge device, the research in the field of on-device deep learning not only focuses on the model accuracy but also on the model efficiency, for example, inference latency. There are many attempts to optimize the existing deep learning models for the purpose of deploying them on the edge devices that meet specific application requirements while maintaining high accuracy. Such work not only requires professional knowledge but also needs a lot of experiments, which limits the customization of neural networks for varied devices and application scenarios. In order to reduce the human intervention in designing and optimizing the neural network structure, multi-objective neural architecture search methods that can automatically search for neural networks featured with high accuracy and can satisfy certain hardware performance requirements are proposed. However, the current methods commonly set accuracy and inference latency as the performance indicator during the search process, and sample numerous network structures to obtain the required neural network. Lacking regulation to the search direction with the search objectives will generate a large number of useless networks during the search process, which influences the search efficiency to a great extent. Therefore, in this paper, an efficient resource-aware search method is proposed. Firstly, the network inference consumption profiling model for any specific device is established, and it can help us directly obtain the resource consumption of each operation in the network structure and the inference latency of the entire sampled network. Next, on the basis of the Bayesian search, a resource-aware Pareto Bayesian search is proposed. Accuracy and inference latency are set as the constraints to regulate the search direction. With a clearer search direction, the overall search efficiency will be improved. Furthermore, cell-based structure and lightweight operation are applied to optimize the search space for further enhancing the search efficiency. The experimental results demonstrate that with our method, the inference latency of the searched network structure reduced 94.71% without scarifying the accuracy. At the same time, the search efficiency increased by 18.18%.

CroApp: A CNN-Based Resource Optimization Approach in Edge Computing Environment

Extendable Multi-Device Collaborative Pipeline Parallel Inference in the Edge-Cloud Scenario

Condense: A Framework for Device and Frequency Adaptive Neural Network Models on the Edge.

Optimizing the Learning Performance in Mobile Augmented Reality Systems with CNN

Automated Exploration and Implementation of Distributed CNN Inference at the Edge

Optimizing the Learning Accuracy in Mobile Augmented Reality Systems with CNN

Researching the CNN Collaborative Inference Mechanism for Heterogeneous Edge Devices

EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters

Rotating Kernel CNN Optimization for Efficient IoT Surveillance on Low-Power Devices

Modeling the Resource Requirements of Convolutional Neural Networks on Mobile Devices

Efficient Resource-Aware Convolutional Neural Architecture Search for Edge Computing with Pareto-Bayesian Optimization

A Unified Optimization Approach for CNN Model Inference on Integrated GPUs

Optimizing Convolutional Neural Network Architectures

Latency-Memory Optimized Splitting of Convolution Neural Networks for Resource Constrained Edge Devices

Multi-Compression Scale DNN Inference Acceleration based on Cloud-Edge-End Collaboration

DeeperThings: Fully Distributed CNN Inference on Resource-Constrained Edge Devices

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

A Cloud-Edge Collaboration Framework for Cognitive Service.

Rethinking Energy-Efficiency of Heterogeneous Computing for CNN-Based Mobile Applications

DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices

Crane: Mitigating Accelerator Under-utilization Caused by Sparsity Irregularities in CNNs