Abstract:Pruning convolutional neural networks offers a promising solution to mitigate the computational complexity challenges encountered during application deployment. However, prevalent pruning techniques primarily concentrate on model parameters or feature mapping analysis to devise static pruning strategies, often overlooking the underlying feature extraction capacity of convolutional kernels. To address this, the study first quantitatively expresses the feature extraction capability of convolutional channels from three aspects: global features, distribution metrics, and directional metrics. It explores the multi-dimensional information of the channels, calculates the overall expectation, variance, and cosine distance from the unit vector as the quantitative results of the channels. Subsequently, a clustering algorithm is employed to categorize the multidimensional information. This approach ensures that convolutional channels grouped within each cluster possess similar feature extraction capabilities. An enhanced differential evolutionary algorithm is utilized to optimize the number of clustering centers across all convolutional layers, ensuring optimal grouping. The final step involves achieving channel sparsification through the calculation of crowding distances for each sample within its designated cluster. This preserves a diverse subset of channels that are critical for maintaining model accuracy. Extensive empirical evaluations conducted on three benchmark image classification datasets demonstrate the efficacy of this method. For instance, on the ImageNet dataset, the ResNet-50 model experiences a substantial reduction in FLOPs by 58.43% while incurring a minimal decrease in TOP-1 accuracy of only 1.15%.

A Single-sample Pruning and Clustering Method for Neural Network Verification

Improving Neural Network Verification through Spurious Region Guided Refinement.

Class-Aware Pruning for Efficient Neural Networks

Structured Deep Neural Network Pruning by Varying Regularization Parameters.

Pruning and Slicing Neural Networks using Formal Verification

Expediting Neural Network Verification via Network Reduction

VeriPrune: Equivalence verification of node pruned neural network

Adversarial Structured Neural Network Pruning

A Dual Relaxation Method for Neural Network Verification

A Clustering Pruning Method Based on Multidimensional Channel Information

A Probabilistic Approach to Neural Network Pruning

FVW: Finding Valuable Weight on Deep Neural Network for Model Pruning

Efficient verification of neural networks based on neuron branching and LP abstraction

Multiple Sparse Spaces Network Pruning Via a Joint Similarity Criterion

Exploiting Channel Similarity for Network Pruning.

Network Pruning Using Sparse Learning and Genetic Algorithm

Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee

Bound Tightening using Rolling-Horizon Decomposition for Neural Network Verification

Efficient Complete Verification of Neural Networks Via Layerwised Splitting and Refinement

Eager Falsification for Accelerating Robustness Verification of Deep Neural Networks

Manifold Regularized Dynamic Network Pruning