Abstract:Convolutional neural network (CNN) pruning is a technique used to remove redundant parameters from the network. By doing so, it aims to greatly reduce the computational complexity and scale of the network while still preserving its accuracy. In the CNN, the majority of parameters are weights that form filters. When it comes to pruning, it is more effective to focus on removing redundant filters rather than insignificant weights within filters. The essence of filter pruning lies in determining the significance or contribution of each filter. Filters that have a significant contribution are kept, while others are pruned. Current methods for calculating contribution in pruning often rely on weight magnitude or filter similarity. However, approaches based solely on assume that small weights are unimportant and ignore correlation between filters, which leads to a significant loss of network accuracy. Those based on filter similarity flatten filter tensors into a vector when calculating filter similarity, and lose the important structural information of filters, or the superposition information of the weight convolution in the corresponding space position. These limitations can compromise the accuracy and effectiveness of the pruning process. This paper proposes an adaptive CNN pruning method based on the structural similarity of filters (APSSF) by taking both the structural characteristics of and the correlation between filters into the consideration for pruning filters. APSSF efficiently calculates the distance between the filters by factoring in information from all the dimensions of filters, and clusters the filters according to the distance threshold determined adaptively according to the compression rate, and deletes a certain number of filters from each category. On the CIFAR10 and ImageNet datasets, APSSF outperforms several state-of-the-art methods. On the CIFAR100, APSSF reduces parameters of networks by 91.71% and 74.80% on VGG-16 and ResNet-34, respectively. The accuracy was decreased only by 0.03 on VGG-16, while on ResNet-34, it was increased by 0.04.

Convolutional neural network acceleration algorithm based on filters pruning

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Structured Probabilistic Pruning for Convolutional Neural Network Acceleration.

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Accelerating Convolutional Neural Networks By Group-Wise 2d-Filter Pruning

Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks

Accelerating Convolutional Networks via Global & Dynamic Filter Pruning

An Entropy-based Pruning Method for CNN Compression

A Pruning Method Based On Feature Abstraction Capability Of Filters

Pruning filters with L1-norm and standard deviation for CNN compression

Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks

Compressing Convolutional Neural Networks Via Factorized Convolutional Filters

APSSF: Adaptive CNN Pruning Based on Structural Similarity of Filters

Accelerating Deep Convnets Via Sparse Subspace Clustering

Channel Pruning Based on Mean Gradient for Accelerating Convolutional Neural Networks

Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks

Pruning Convolutional Neural Networks via Stochastic Gradient Hard Thresholding

Pruning Convolutional Neural Networks Via Filter Similarity Analysis.

Incremental Filter Pruning Via Random Walk for Accelerating Deep Convolutional Neural Networks

Efficient CNNs via Passive Filter Pruning