Abstract:Neural network pruning provides significant performance in reducing the resource requirements for deploying deep convolutional models. Recent pruning techniques concentrate on eliminating less important or redundant channels from the network. However, these well-designed methods conflict in some situations. For example, some filters are important in importance-based methods but may be regarded as redundant in similarity-based methods. So, the correctness of some existing methods is questionable. In this paper, a novel pruning approach, entitled weight-adaptive channel pruning (WACP), is presented to address the problem. Our approach takes full advantage of the feature similarity information instead of simply categorizing the similarity feature as redundant. Specifically, we first reveal that there is a stable similarity relationship between different output features, independent of the batch size of input images. Then, based on the similarity information, we propose a weight-adaptive compensation strategy to minimize the performance loss caused by pruning. Moreover, we design a novel channel pruning algorithm that determines which features should be retained from a set of similar features by introducing the closeness centrality of graph theory. Extensive and targeted experiments have demonstrated the validity of our proposed WACP for compressing networks. The comparison results demonstrate that the WACP achieves state-of-the-art performance on several benchmark networks and datasets, even for a very high compression rate. For example, WACP improves accuracy by 0.46% while reducing FLOPs by 52.2% and parameters by 43.5% with ResNet-56 on CIFAR-10. For ResNet-50 on ImageNet, WACP prunes more than 55% of FLOPs with only a 0.70%/0.42% decline in top-1/top-5 accuracy. The codes are at https://github.com/lsianke/WACP .

Fuzzy Pruning for Compression of Convolutional Neural Networks.

Class-Aware Pruning for Efficient Neural Networks

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Filter Pruning Via Feature Map Clustering.

Filter Pruning with a Feature Map Entropy Importance Criterion for Convolution Neural Networks Compressing

NFP: A No Fine-tuning Pruning Approach for Convolutional Neural Network Compression

Convolutional neural network acceleration algorithm based on filters pruning

Compressing Convolutional Neural Networks Via Factorized Convolutional Filters

Weight-adaptive Channel Pruning for CNNs Based on Closeness-Centrality Modeling

Pruning filters with L1-norm and standard deviation for CNN compression

Pruning Convolutional Neural Networks Via Filter Similarity Analysis.

A Sensitivity-based Pruning Method for Convolutional Neural Networks.

Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks

Compression of Convolutional Neural Networks With Divergent Representation of Filters

Frequency-Domain Dynamic Pruning for Convolutional Neural Networks

Optimization Based Layer-Wise Pruning Threshold Method for Accelerating Convolutional Neural Networks

Functionality-Oriented Convolutional Filter Pruning

An Entropy-based Pruning Method for CNN Compression

Convolution Network Pruning Based on the Evaluation of the Importance of Characteristic Attributions

Structured Pruning Via Feature Channels Similarity and Mutual Learning for Convolutional Neural Network Compression