Abstract:Deploying deep neural networks on mobile devices is a challenging task. Current model compression methods such as matrix decomposition effectively reduce the deployed model size, but still cannot satisfy real-time processing requirement. This paper first discovers that the major obstacle is the excessive execution time of non-tensor layers such as pooling and normalization without tensor-like trainable parameters. This motivates us to design a novel acceleration framework: DeepRebirth through "slimming" existing consecutive and parallel non-tensor and tensor layers. The layer slimming is executed at different substructures: (a) streamline slimming by merging the consecutive non-tensor and tensor layer vertically; (b) branch slimming by merging non-tensor and tensor branches horizontally. The proposed optimization operations significantly accelerate the model execution and also greatly reduce the run-time memory cost since the slimmed model architecture contains less hidden layers. To maximally avoid accuracy loss, the parameters in new generated layers are learned with layer-wise fine-tuning based on both theoretical analysis and empirical verification. As observed in the experiment, DeepRebirth achieves more than 3x speed-up and 2.5x run-time memory saving on GoogLeNet with only 0.4% drop on top-5 accuracy in ImageNet. Furthermore, by combining with other model compression techniques, DeepRebirth offers an average of 106.3ms inference time on the CPU of Samsung Galaxy S5 with 86.5% top-5 accuracy, 14% faster than SqueezeNet which only has a top-5 accuracy of 80.5%.

Lightweight Convolutional Representations for On-Device Natural Language Processing

Explore Training of Deep Convolutional Neural Networks on Battery-powered Mobile Devices: Design and Application

Condense: A Framework for Device and Frequency Adaptive Neural Network Models on the Edge.

Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications

Lightweight compression of neural network feature tensors for collaborative intelligence

Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

Extreme Model Compression for On-device Natural Language Understanding

DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices

Extract Generalization Ability From Convolutional Neural Networks

Compression of end-to-end non-autoregressive image-to-speech system for low-resourced devices

Fast Video Facial Expression Recognition by a Deeply Tensor-Compressed LSTM Neural Network for Mobile Devices

High performance ultra-low-precision convolutions on mobile devices

Smart-DNN: Efficiently Reducing the Memory Requirements of Running Deep Neural Networks on Resource-constrained Platforms

Lightweight Deep Learning for Resource-Constrained Environments: A Survey

LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution

Weightless: Lossy Weight Encoding For Deep Neural Network Compression

Convolutional neural networks compression with low rank and sparse tensor decompositions

High Performance Depthwise and Pointwise Convolutions on Mobile Devices

Face Recognition Based on Lightweight Convolutional Neural Networks