Abstract:Neural network pruning offers a promising prospect to facilitate deploying deep neural networks on resource-limited devices. However, existing methods are still challenged by the training inefficiency and labor cost in pruning designs, due to missing theoretical guidance of non-salient network components. In this paper, we propose a novel filter pruning method by exploring the High Rank of feature maps (HRank). Our HRank is inspired by the discovery that the average rank of multiple feature maps generated by a single filter is always the same, regardless of the number of image batches CNNs receive. Based on HRank, we develop a method that is mathematically formulated to prune filters with low-rank feature maps. The principle behind our pruning is that low-rank feature maps contain less information, and thus pruned results can be easily reproduced. Besides, we experimentally show that weights with high-rank feature maps contain more important information, such that even when a portion is not updated, very little damage would be done to the model performance. Without introducing any additional constraints, HRank leads to significant improvements over the state-of-the-arts in terms of FLOPs and parameters reduction, with similar accuracies. For example, with ResNet-110, we achieve a 58.2%-FLOPs reduction by removing 59.2% of the parameters, with only a small loss of 0.14% in top-1 accuracy on CIFAR-10. With Res-50, we achieve a 43.8%-FLOPs reduction by removing 36.7% of the parameters, with only a loss of 1.17% in the top-1 accuracy on ImageNet. The codes can be available at https://github.com/1mbxmu/HRank.

A Filter Pruning Method of CNN Models Based on Feature Maps Clustering

Filter Pruning Via Feature Map Clustering.

Towards Efficient Filter Pruning Via Topology

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

A Graphical Approach for Filter Pruning by Exploring the Similarity Relation between Feature Maps

Convolutional neural network acceleration algorithm based on filters pruning

A Pruning Method Based On Feature Abstraction Capability Of Filters

CSHE: Network Pruning by Using Cluster Similarity and Matrix Eigenvalues

Feature Statistics Guided Efficient Filter Pruning

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Online Filter Clustering and Pruning for Efficient Convnets

Pruning filters with L1-norm and standard deviation for CNN compression

Filter Pruning with a Feature Map Entropy Importance Criterion for Convolution Neural Networks Compressing

HRank: Filter Pruning Using High-Rank Feature Map.

Filter Pruning-Based Two-Step Feature Map Reconstruction

Filter Pruning for CNN with Enhanced Linear Representation Redundancy

Online Filter Weakening and Pruning for Efficient Convnets

Asymptotic Soft Cluster Pruning for Deep Neural Networks

Filter Pruning by Switching to Neighboring CNNs With Good Attributes