Abstract:Convolutional neural network (CNN) pruning is a technique used to remove redundant parameters from the network. By doing so, it aims to greatly reduce the computational complexity and scale of the network while still preserving its accuracy. In the CNN, the majority of parameters are weights that form filters. When it comes to pruning, it is more effective to focus on removing redundant filters rather than insignificant weights within filters. The essence of filter pruning lies in determining the significance or contribution of each filter. Filters that have a significant contribution are kept, while others are pruned. Current methods for calculating contribution in pruning often rely on weight magnitude or filter similarity. However, approaches based solely on assume that small weights are unimportant and ignore correlation between filters, which leads to a significant loss of network accuracy. Those based on filter similarity flatten filter tensors into a vector when calculating filter similarity, and lose the important structural information of filters, or the superposition information of the weight convolution in the corresponding space position. These limitations can compromise the accuracy and effectiveness of the pruning process. This paper proposes an adaptive CNN pruning method based on the structural similarity of filters (APSSF) by taking both the structural characteristics of and the correlation between filters into the consideration for pruning filters. APSSF efficiently calculates the distance between the filters by factoring in information from all the dimensions of filters, and clusters the filters according to the distance threshold determined adaptively according to the compression rate, and deletes a certain number of filters from each category. On the CIFAR10 and ImageNet datasets, APSSF outperforms several state-of-the-art methods. On the CIFAR100, APSSF reduces parameters of networks by 91.71% and 74.80% on VGG-16 and ResNet-34, respectively. The accuracy was decreased only by 0.03 on VGG-16, while on ResNet-34, it was increased by 0.04.

BAP: Bilateral Asymptotic Pruning for Optimizing CNNs on Image Tasks

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

Structured Deep Neural Network Pruning by Varying Regularization Parameters.

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Global balanced iterative pruning for efficient convolutional neural networks

LAPP: Layer Adaptive Progressive Pruning for Compressing CNNs from Scratch

Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks

Iterative clustering pruning for convolutional neural networks

An Efficient Channel-level Pruning for CNNs without Fine-tuning

Exploiting Channel Similarity for Network Pruning.

Automatic channel pruning via clustering and swarm intelligence optimization for CNN

AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance

APSSF: Adaptive CNN Pruning Based on Structural Similarity of Filters

Pruning filters with L1-norm and standard deviation for CNN compression

Filter Pruning for CNN with Enhanced Linear Representation Redundancy

NFP: A No Fine-tuning Pruning Approach for Convolutional Neural Network Compression

AACP: Model Compression by Accurate and Automatic Channel Pruning.