Abstract:Convolutional neural network (CNN) pruning is a technique used to remove redundant parameters from the network. By doing so, it aims to greatly reduce the computational complexity and scale of the network while still preserving its accuracy. In the CNN, the majority of parameters are weights that form filters. When it comes to pruning, it is more effective to focus on removing redundant filters rather than insignificant weights within filters. The essence of filter pruning lies in determining the significance or contribution of each filter. Filters that have a significant contribution are kept, while others are pruned. Current methods for calculating contribution in pruning often rely on weight magnitude or filter similarity. However, approaches based solely on assume that small weights are unimportant and ignore correlation between filters, which leads to a significant loss of network accuracy. Those based on filter similarity flatten filter tensors into a vector when calculating filter similarity, and lose the important structural information of filters, or the superposition information of the weight convolution in the corresponding space position. These limitations can compromise the accuracy and effectiveness of the pruning process. This paper proposes an adaptive CNN pruning method based on the structural similarity of filters (APSSF) by taking both the structural characteristics of and the correlation between filters into the consideration for pruning filters. APSSF efficiently calculates the distance between the filters by factoring in information from all the dimensions of filters, and clusters the filters according to the distance threshold determined adaptively according to the compression rate, and deletes a certain number of filters from each category. On the CIFAR10 and ImageNet datasets, APSSF outperforms several state-of-the-art methods. On the CIFAR100, APSSF reduces parameters of networks by 91.71% and 74.80% on VGG-16 and ResNet-34, respectively. The accuracy was decreased only by 0.03 on VGG-16, while on ResNet-34, it was increased by 0.04.

Symmetric Convolutional Filters: A Novel Way to Constrain Parameters in CNN

SparseConnect: Regularising CNNs on Fully Connected Layers

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Centrosymmetric constrained Convolutional Neural Networks

Efficient Similarity-based Passive Filter Pruning for Compressing CNNs

Overfitting Remedy by Sparsifying Regularization on Fully-Connected Layers of CNNs.

APSSF: Adaptive CNN Pruning Based on Structural Similarity of Filters

Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNs

Adversarial Structured Neural Network Pruning

Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images

Convolutional Neural Network Compression via Dynamic Parameter Rank Pruning

Compressing Convolutional Neural Networks Via Factorized Convolutional Filters

Symmetry Defense Against CNN Adversarial Perturbation Attacks

Low-Cost Parameterizations of Deep Convolutional Neural Networks

FSNet: Compression of Deep Convolutional Neural Networks by Filter Summary

Feature Statistics Guided Efficient Filter Pruning

Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks

A Passive Similarity based CNN Filter Pruning for Efficient Acoustic Scene Classification

Provable Filter Pruning for Efficient Neural Networks

A Pre-defined Sparse Kernel Based Convolution for Deep CNNs