Abstract:Filter pruning is widely used for inference acceleration and compatibility with off-the-shelf hardware devices. Some filter pruning methods have proposed various criteria to approximate the importance of filters, and then sort the filters globally or locally to prune the redundant parameters. However, the current criterion-based methods have problems: (1) parameters with smaller criterion values for extracting edge features are easily ignored, and (2) there is a strong correlation between different criteria, resulting in similar pruning structures. In this article, we propose a novel simple but effective pruning method based on filter similarity, which is used to evaluate the similarity between filters instead of the importance of a single filter. The proposed method first calculates the similarity of the filters pairwise in one convolutional layer and then obtains the similarity distribution. Finally, the filters with high similarity to others are deleted from the distribution or set to zero. In addition, the proposed algorithm does not need to specify the pruning rate for each layer, and only needs to set the desired FLOPs or parameter reduction to obtain the final compression model. We also provide iterative pruning strategies for hard pruning and soft pruning to satisfy the tradeoff requirements of accuracy and memory in different scenarios. Extensive experiments on various representative benchmark datasets across different network architectures demonstrate the effectiveness of our proposed method. For example, on CIFAR10, the proposed algorithm achieves 61.1% FLOPs reduction by removing 58.3% of the parameters, with no loss in Top-1 accuracy on ResNet-56; and reduces 53.05% FLOPs on ResNet-50 with only 0.29% Top-1 accuracy degradation on ILSVRC-2012.

Filter Sketch for Network Pruning

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Towards Efficient Filter Pruning Via Topology

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Class-Aware Pruning for Efficient Neural Networks

Filter Pruning Via Feature Map Clustering.

Network Pruning Using Adaptive Exemplar Filters

Network Pruning Spaces

A Graphical Approach for Filter Pruning by Exploring the Similarity Relation between Feature Maps

Network Compression via Central Filter

Non-Parametric Adaptive Network Pruning

Network Pruning Via Probing the Importance of Filters

Pruning Filter in Filter

Dynamic Network Pruning Via Filter Attention Mechanism and Feature Scaling Factor

SNF: Filter Pruning via Searching the Proper Number of Filters

Pruning Networks with Cross-Layer Ranking K-Reciprocal Nearest Filters

Filter Pruning Via Separation of Sparsity Search and Model Training

Model pruning based on filter similarity for edge device deployment

Learning Compact ConvNets Through Filter Pruning Based on the Saliency of a Feature Map.

Online Filter Weakening and Pruning for Efficient Convnets