Abstract:The success of convolutional neural networks (CNNs) in computer vision applications has been accompanied by a significant increase of computation and memory costs, which prohibits their usage on resource-limited environments, such as mobile systems or embedded devices. To this end, the research of CNN compression has recently become emerging. In this paper, we propose a novel filter pruning scheme, termed structured sparsity regularization (SSR), to simultaneously speed up the computation and reduce the memory overhead of CNNs, which can be well supported by various off-the-shelf deep learning libraries. Concretely, the proposed scheme incorporates two different regularizers of structured sparsity into the original objective function of filter pruning, which fully coordinates the global output and local pruning operations to adaptively prune filters. We further propose an alternative updating with Lagrange multipliers (AULM) scheme to efficiently solve its optimization. AULM follows the principle of alternating direction method of multipliers (ADMM) and alternates between promoting the structured sparsity of CNNs and optimizing the recognition loss, which leads to a very efficient solver (2.5x to the most recent work that directly solves the group sparsity-based regularization). Moreover, by imposing the structured sparsity, the online inference is extremely memory-light since the number of filters and the output feature maps are simultaneously reduced. The proposed scheme has been deployed to a variety of state-of-the-art CNN structures, including LeNet, AlexNet, VGGNet, ResNet, and GoogLeNet, over different data sets. Quantitative results demonstrate that the proposed scheme achieves superior performance over the state-of-the-art methods. We further demonstrate the proposed compression scheme for the task of transfer learning, including domain adaptation and object detection, which also show exciting performance gains over the state-of-the-art filter pruning methods.

Adaptive Structured Sparse Network for Efficient CNNs with Feature Regularization.

Adaptive Structured Sparse Network for Efficient CNNs with Feature Regularization

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Adaptive Pixel-wise Structured Sparse Network for Efficient CNNs

SUBP: Soft Uniform Block Pruning for 1 X N Sparse CNNs Multithreading Acceleration

SUBP: Soft Uniform Block Pruning for 1xn Sparse CNNs Multithreading Acceleration

SparseConnect: Regularising CNNs on Fully Connected Layers

Structured Deep Neural Network Pruning by Varying Regularization Parameters.

Efficient Structure Slimming for Spiking Neural Networks

Joint Sparsity with Mixed Granularity for Efficient GPU Implementation

Structured Feature Sparsity Training for Convolutional Neural Network Compression

Learning Structured Sparsity in Deep Neural Networks

Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning

Overfitting Remedy by Sparsifying Regularization on Fully-Connected Layers of CNNs.

Adversarial Structured Neural Network Pruning

Efficient Neural Networks with Spatial Wise Sparsity Using Unified Importance Map.

SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training

A Software/Hardware Co-design Local Irregular Sparsity Method for Accelerating CNNs on FPGA.

Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Learning Sparse Patterns in Deep Neural Networks

Performance of Training Sparse Deep Neural Networks on GPUs