Abstract:Modern Convolutional Neural Networks (CNNs) are usually restricted by their massive computation and high storage. Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance degradation. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fragility of the expressiveness of CNNs, and thus calls for a more gentle regularization scheme so that the networks can adapt during pruning. To achieve this, we propose a novel regularization-based pruning method, named IncReg, to incrementally assign different regularization factors to different weights based on their relative importance. Empirical analysis on CIFAR-10 dataset verifies the merits of IncReg. Further extensive experiments with popular CNNs on CIFAR-10 and ImageNet datasets show that IncReg achieves comparable to even better results compared with state-of-the-arts. Moreover, to resolve the problem that column pruning cannot be directly applied to off-the-shelf deep learning libraries for acceleration, we generalize IncReg from column pruning to spatial pruning, which can equip existing structured pruning methods (such as channel pruning) for further acceleration with ignorable accuracy loss. Our source codes and trained models are available at: https://github.com/mingsun-tse/caffe_increghttps://github.com/mingsun-tse/caffe_increg.

Accelerating Convolutional Neural Networks By Group-Wise 2d-Filter Pruning

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Structured Probabilistic Pruning for Convolutional Neural Network Acceleration.

Convolutional neural network acceleration algorithm based on filters pruning

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Structured Deep Neural Network Pruning by Varying Regularization Parameters.

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

Towards Efficient Filter Pruning Via Topology

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Accelerating Convolutional Networks via Global & Dynamic Filter Pruning

Online Filter Weakening and Pruning for Efficient Convnets

Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks

Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

Complex hybrid weighted pruning method for accelerating convolutional neural networks

A One-step Pruning-recovery Framework for Acceleration of Convolutional Neural Networks

Pruning filters with L1-norm and standard deviation for CNN compression

Pruning 3D Filters For Accelerating 3D ConvNets

Learning to Prune Filters in Convolutional Neural Networks

Compressing Convolutional Neural Networks Via Factorized Convolutional Filters

Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration.

Improve Convolutional Neural Network Pruning by Maximizing Filter Variety