Abstract:As convolutional neural networks (CNNs) have become increasingly deeper in recent years, the requirements for the quantity of data and hardware resources have gradually increased. CNN also reveals salient redundancy in several tasks. The existing magnitude-based pruning methods are efficient, but the performance of the compressed network is unpredictable. While the accuracy loss after pruning based on the structure sensitivity is relatively slight, the process is time-consuming, and the algorithm complexity is notable. To fully combine the advantages of the two types of methods, we propose a novel automatic channel pruning method (ACP). Specifically, we first perform layerwise channel clustering via the similarity of the feature maps to perform preliminary pruning on the network. Then, a population initialization method is introduced to transform the pruned structure into a candidate population. Finally, we conduct iterative searching and optimization based on particle swarm optimization (PSO) to find the optimal compressed structure. The compact network is then retrained to mitigate the accuracy loss from pruning. Our method is evaluated against several state-of-the-art CNNs on three different classification datasets CIFAR-10/100 and ILSVRC-2012. On the ILSVRC-2012, when removing 64.36% parameters and 63.34% floating-point operations (FLOPs) of ResNet-50, the Top-1 and Top-5 accuracy drops are less than 0.9%. Moreover, we demonstrate that without harming the overall performance, it is possible to compress the SSD by more than 50% on the target detection dataset PASCAL VOC. This further verifies that the proposed method can also be applied to other CNNs and application scenarios.

Convolutional neural network pruning based on multi-objective feature map selection for image classification

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Class-Aware Pruning for Efficient Neural Networks

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Filter Pruning Via Feature Map Clustering.

Cross-layer importance evaluation for neural network pruning

Multi-Objective Compression for CNNs via Evolutionary Algorithm

Design a multi-objective optimization with dynamic and global filter pruning strategy for convolutional neural network

Compressing CNNs Using Multilevel Filter Pruning for the Edge Nodes of Multimedia Internet of Things

Iterative clustering pruning for convolutional neural networks

Automatic channel pruning via clustering and swarm intelligence optimization for CNN

Global balanced iterative pruning for efficient convolutional neural networks

Adaptive Scaling Filter Pruning Method for Vision Networks With Embedded Devices

SNPF: Sensitiveness Based Network Pruning Framework for Efficient Edge Computing

Pruning Deep Convolutional Neural Network Using Conditional Mutual Information

NFP: A No Fine-tuning Pruning Approach for Convolutional Neural Network Compression

GenExp: Multi-objective pruning for deep neural network based on genetic algorithm

Multi-Objective Net Architecture Pruning for Remote Sensing Classification.