Abstract:Convolutional neural network (CNN) pruning is a technique used to remove redundant parameters from the network. By doing so, it aims to greatly reduce the computational complexity and scale of the network while still preserving its accuracy. In the CNN, the majority of parameters are weights that form filters. When it comes to pruning, it is more effective to focus on removing redundant filters rather than insignificant weights within filters. The essence of filter pruning lies in determining the significance or contribution of each filter. Filters that have a significant contribution are kept, while others are pruned. Current methods for calculating contribution in pruning often rely on weight magnitude or filter similarity. However, approaches based solely on assume that small weights are unimportant and ignore correlation between filters, which leads to a significant loss of network accuracy. Those based on filter similarity flatten filter tensors into a vector when calculating filter similarity, and lose the important structural information of filters, or the superposition information of the weight convolution in the corresponding space position. These limitations can compromise the accuracy and effectiveness of the pruning process. This paper proposes an adaptive CNN pruning method based on the structural similarity of filters (APSSF) by taking both the structural characteristics of and the correlation between filters into the consideration for pruning filters. APSSF efficiently calculates the distance between the filters by factoring in information from all the dimensions of filters, and clusters the filters according to the distance threshold determined adaptively according to the compression rate, and deletes a certain number of filters from each category. On the CIFAR10 and ImageNet datasets, APSSF outperforms several state-of-the-art methods. On the CIFAR100, APSSF reduces parameters of networks by 91.71% and 74.80% on VGG-16 and ResNet-34, respectively. The accuracy was decreased only by 0.03 on VGG-16, while on ResNet-34, it was increased by 0.04.

Model pruning based on filter similarity for edge device deployment

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Towards Efficient Filter Pruning Via Topology

Class-Aware Pruning for Efficient Neural Networks

Filter Pruning Via Feature Map Clustering.

A Graphical Approach for Filter Pruning by Exploring the Similarity Relation between Feature Maps

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Convolutional neural network acceleration algorithm based on filters pruning

QSFM: Model Pruning Based on Quantified Similarity Between Feature Maps for AI on Edge.

Optimizing deep neural networks on intelligent edge accelerators via flexible-rate filter pruning

Similarity Based Filter Pruning for Efficient Super-Resolution Models

A Filter Rank Based Pruning Method for Convolutional Neural Networks

Non-Parametric Adaptive Network Pruning

Pruning Convolutional Neural Networks Via Filter Similarity Analysis.

Filter Pruning Via Separation of Sparsity Search and Model Training

Network Pruning Using Adaptive Exemplar Filters

APSSF: Adaptive CNN Pruning Based on Structural Similarity of Filters

Filter Pruning with a Feature Map Entropy Importance Criterion for Convolution Neural Networks Compressing

Network Pruning Via Probing the Importance of Filters

Learning Compact ConvNets Through Filter Pruning Based on the Saliency of a Feature Map.

Cross-layer importance evaluation for neural network pruning