Abstract:Deep convolutional neural networks (CNNs) have achieved impressive performance in many computer vision tasks. However, their large model sizes require heavy computational resources, making pruning redundant filters from existing pre-trained CNNs an essential task in developing efficient models for resource-constrained devices. Whole-network filter pruning algorithms prune varying fractions of filters from each layer, hence providing greater flexibility. Current whole-network pruning methods are either computationally expensive due to the need to calculate the loss for each pruned filter using a training dataset, or use various heuristic / learned criteria for determining the pruning fractions for each layer. This paper proposes a two-level hierarchical approach for whole-network filter pruning which is efficient and uses the classification loss as the final criterion. The lower-level algorithm (called filter-pruning) uses a sparse-approximation formulation based on linear approximation of filter weights. We explore two algorithms: orthogonal matching pursuit-based greedy selection and a greedy backward pruning approach. The backward pruning algorithm uses a novel closed-form error criterion for efficiently selecting the optimal filter at each stage, thus making the whole algorithm much faster. The higher-level algorithm (called layer-selection) greedily selects the best-pruned layer (pruning using the filter-selection algorithm) using a global pruning criterion. We propose algorithms for two different global-pruning criteria: (1) layer-wise relative error (HBGS), and (2) final classification error (HBGTS). Our suite of algorithms outperforms state-of-the-art pruning methods on ResNet18, ResNet32, ResNet56, VGG16, and ResNext101. Our method reduces the RAM requirement for ResNext101 from 7.6 GB to 1.5 GB and achieves a 94% reduction in FLOPS without losing accuracy on CIFAR-10.

Towards Efficient Filter Pruning Via Topology

Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Filter Pruning Via Feature Map Clustering.

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Class-Aware Pruning for Efficient Neural Networks

Network Pruning Via Probing the Importance of Filters

A Graphical Approach for Filter Pruning by Exploring the Similarity Relation between Feature Maps

Network Pruning Spaces

HRank: Filter Pruning Using High-Rank Feature Map.

Pruning the Deep Neural Network by Similar Function

Filter Pruning by Switching to Neighboring CNNs With Good Attributes

Functionality-Oriented Convolutional Filter Pruning

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Model pruning based on filter similarity for edge device deployment

Online Filter Weakening and Pruning for Efficient Convnets

Provable Filter Pruning for Efficient Neural Networks

A Greedy Hierarchical Approach to Whole-Network Filter- Pruning in CNNs

Pruning filters with L1-norm and standard deviation for CNN compression

Filter Pruning based on Information Capacity and Independence