Abstract:Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning regularity: non-structured, fine-grained pruning can achieve high sparsity and accuracy, but is not hardware friendly; structured, coarse-grained pruning exploits hardware-efficient structures in pruning, but suffers from accuracy drop when the pruning rate is high. In this paper, we introduce PCONV, comprising a new sparsity dimension, – fine-grained pruning patterns inside the coarse-grained structures. PCONV comprises two types of sparsities, Sparse Convolution Patterns (SCP) which is generated from intra-convolution kernel pruning and connectivity sparsity generated from inter-convolution kernel pruning. Essentially, SCP enhances accuracy due to its special vision properties, and connectivity sparsity increases pruning rate while maintaining balanced workload on filter computation. To deploy PCONV, we develop a novel compiler-assisted DNN inference framework and execute PCONV models in real-time without accuracy compromise, which cannot be achieved in prior work. Our experimental results show that, PCONV outperforms three state-of-art end-to-end DNN frameworks, TensorFlow-Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 39.2 ×, 11.4 ×, and 6.3 ×, respectively, with no accuracy loss. Mobile devices can achieve real-time inference on large-scale DNNs.

CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

Condense: A Framework for Device and Frequency Adaptive Neural Network Models on the Edge.

CondenseNet V2: Sparse Feature Reactivation for Deep Networks

Effnet: An Efficient Structure for Convolutional Neural Networks

Efficient densely connected convolutional neural networks

Convolutional Networks with Dense Connectivity

ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices

ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions

Efficient Network Compression Through Smooth-Lasso Constraint

Grouped Pointwise Convolutions Significantly Reduces Parameters in EfficientNet

CondConv: Conditionally Parameterized Convolutions for Efficient Inference

Structured Convolutions for Efficient Neural Network Design

HENet:A Highly Efficient Convolutional Neural Networks Optimized for Accuracy, Speed and Storage

Learning Efficient Convolutional Networks Through Network Slimming.

LDCNet: A Lightweight Multi-Scale Convolutional Neural Network Using Local Dense Connectivity for Image Recognition

LA-ShuffleNet: A Strong Convolutional Neural Network for Edge Computing Devices

Highly shared Convolutional Neural Networks

Image Classification with CondenseNeXt for ARM-Based Computing Platforms

PCONV: the Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices.

SlimConv: Reducing Channel Redundancy in Convolutional Neural Networks by Features Recombining

VarGNet: Variable Group Convolutional Neural Network for Efficient Embedded Computing