Abstract:Convolutional neural network (CNN) pruning is a technique used to remove redundant parameters from the network. By doing so, it aims to greatly reduce the computational complexity and scale of the network while still preserving its accuracy. In the CNN, the majority of parameters are weights that form filters. When it comes to pruning, it is more effective to focus on removing redundant filters rather than insignificant weights within filters. The essence of filter pruning lies in determining the significance or contribution of each filter. Filters that have a significant contribution are kept, while others are pruned. Current methods for calculating contribution in pruning often rely on weight magnitude or filter similarity. However, approaches based solely on assume that small weights are unimportant and ignore correlation between filters, which leads to a significant loss of network accuracy. Those based on filter similarity flatten filter tensors into a vector when calculating filter similarity, and lose the important structural information of filters, or the superposition information of the weight convolution in the corresponding space position. These limitations can compromise the accuracy and effectiveness of the pruning process. This paper proposes an adaptive CNN pruning method based on the structural similarity of filters (APSSF) by taking both the structural characteristics of and the correlation between filters into the consideration for pruning filters. APSSF efficiently calculates the distance between the filters by factoring in information from all the dimensions of filters, and clusters the filters according to the distance threshold determined adaptively according to the compression rate, and deletes a certain number of filters from each category. On the CIFAR10 and ImageNet datasets, APSSF outperforms several state-of-the-art methods. On the CIFAR100, APSSF reduces parameters of networks by 91.71% and 74.80% on VGG-16 and ResNet-34, respectively. The accuracy was decreased only by 0.03 on VGG-16, while on ResNet-34, it was increased by 0.04.

Efficient Structured Pruning Based on Deep Feature Stabilization

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Efficient Block Pruning Based on Kernel and Feature Stablization

Towards Optimal Structured CNN Pruning Via Generative Adversarial Learning

Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning

Structured Pruning of Convolutional Neural Networks Via L1 Regularization

LAPP: Layer Adaptive Progressive Pruning for Compressing CNNs from Scratch

Adversarial Structured Neural Network Pruning

Toward Efficient CNN Compression Via Coordinate Descent Structure Search

Structured Feature Sparsity Training for Convolutional Neural Network Compression

Pruning filters with L1-norm and standard deviation for CNN compression

Global balanced iterative pruning for efficient convolutional neural networks

Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks

APSSF: Adaptive CNN Pruning Based on Structural Similarity of Filters

Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks

Filter Pruning with a Feature Map Entropy Importance Criterion for Convolution Neural Networks Compressing