Abstract:Neural network pruning provides significant performance in reducing the resource requirements for deploying deep convolutional models. Recent pruning techniques concentrate on eliminating less important or redundant channels from the network. However, these well-designed methods conflict in some situations. For example, some filters are important in importance-based methods but may be regarded as redundant in similarity-based methods. So, the correctness of some existing methods is questionable. In this paper, a novel pruning approach, entitled weight-adaptive channel pruning (WACP), is presented to address the problem. Our approach takes full advantage of the feature similarity information instead of simply categorizing the similarity feature as redundant. Specifically, we first reveal that there is a stable similarity relationship between different output features, independent of the batch size of input images. Then, based on the similarity information, we propose a weight-adaptive compensation strategy to minimize the performance loss caused by pruning. Moreover, we design a novel channel pruning algorithm that determines which features should be retained from a set of similar features by introducing the closeness centrality of graph theory. Extensive and targeted experiments have demonstrated the validity of our proposed WACP for compressing networks. The comparison results demonstrate that the WACP achieves state-of-the-art performance on several benchmark networks and datasets, even for a very high compression rate. For example, WACP improves accuracy by 0.46% while reducing FLOPs by 52.2% and parameters by 43.5% with ResNet-56 on CIFAR-10. For ResNet-50 on ImageNet, WACP prunes more than 55% of FLOPs with only a 0.70%/0.42% decline in top-1/top-5 accuracy. The codes are at https://github.com/lsianke/WACP .

Exploiting Channel Similarity for Network Pruning.

Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Learning Compact Networks Via Similarity-Aware Channel Pruning

Weight-adaptive Channel Pruning for CNNs Based on Closeness-Centrality Modeling

CCPrune: Collaborative Channel Pruning for Learning Compact Convolutional Networks

Multiple Sparse Spaces Network Pruning Via a Joint Similarity Criterion

Adversarial Structured Neural Network Pruning

Exploiting similarity-induced redundancies in correlation topology for channel pruning in deep convolutional neural networks

Dynamical Channel Pruning by Conditional Accuracy Change for Deep Neural Networks

Filter Pruning for CNN with Enhanced Linear Representation Redundancy

Network Pruning Using Sparse Learning and Genetic Algorithm

Energy-saving CNN with Clustering Channel Pruning

Loss-Driven Channel Pruning of Convolutional Neural Networks.

Adaptive Pruning of Channel Spatial Dependability in Convolutional Neural Networks

Random pruning: channel sparsity by expectation scaling factor.

ACP: Automatic Channel Pruning Method by Introducing Additional Loss for Deep Neural Networks.

Pruning the Deep Neural Network by Similar Function

An Efficient Channel-level Pruning for CNNs without Fine-tuning

Accelerate CNN Via Recursive Bayesian Pruning.