Abstract:Channel pruning is a promising method for accelerating and compressing convolutional neural networks. However, current pruning algorithms still remain unsolved problems that how to assign layer-wise pruning ratios properly and discard the least important channels with a convincing criterion. In this paper, we present a novel channel pruning approach via information theory and interpretability of neural networks. Specifically, we regard information entropy as the expected amount of information for convolutional layers. In addition, if we suppose a matrix as a system of linear equations, a higher-rank matrix represents there exist more solutions to it, which indicates more uncertainty. From the point of view of information theory, the rank can also describe the amount of information. In a neural network, considering the rank and entropy as two information indicators of convolutional layers, we propose a fusion function to reach a compromise of them, where the fusion results are defined as ``information concentration''. When pre-defining layer-wise pruning ratios, we employ the information concentration as a reference instead of heuristic and engineering tuning to provide a more interpretable solution. Moreover, we leverage Shapley values, which are a potent tool in the interpretability of neural networks, to evaluate the channel contributions and discard the least important channels for model compression while maintaining its performance. Extensive experiments demonstrate the effectiveness and promising performance of our method. For example, our method improves the accuracy by 0.21% when reducing 45.5% FLOPs and removing 40.3% parameters for ResNet-56 on CIFAR-10. Moreover, our method obtains loss in Top-1/Top-5 accuracies of 0.43%/0.11% by reducing 41.6% FLOPs and removing 35.0% parameters for ResNet-50 on ImageNet.

Channel Pruning for Accelerating Convolutional Neural Networks via Wasserstein Metric.

Learning to Slim Deep Networks with Bandit Channel Pruning

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Channel Pruning Based on Mean Gradient for Accelerating Convolutional Neural Networks

Dynamical Channel Pruning by Conditional Accuracy Change for Deep Neural Networks

Weight-adaptive Channel Pruning for CNNs Based on Closeness-Centrality Modeling

Exploiting Channel Similarity for Accelerating Deep Convolutional Neural Networks

CCPrune: Collaborative Channel Pruning for Learning Compact Convolutional Networks

Adaptive Weight Learning for Dynamic Channel Pruning

ACP: Automatic Channel Pruning Method by Introducing Additional Loss for Deep Neural Networks.

Channel Pruning for Accelerating Very Deep Neural Networks

Exploiting Channel Similarity for Network Pruning.

Adaptive Pruning of Channel Spatial Dependability in Convolutional Neural Networks

Discrimination-Aware Channel Pruning For Deep Neural Networks

Loss-Driven Channel Pruning of Convolutional Neural Networks.

Convolutional Neural Network Compression Method Based on Multi-Factor Channel Pruning

Variational Convolutional Neural Network Pruning

Revisiting Random Channel Pruning for Neural Network Compression

Dynamic Channel Pruning with Adaptive Weight Learning

An Effective Information Theoretic Framework for Channel Pruning