Automatic channel pruning via clustering and swarm intelligence optimization for CNN

Jingfei Chang,Yang Lu,Ping Xue,Yiqun Xu,Zhen Wei
DOI: https://doi.org/10.1007/s10489-022-03508-1
IF: 5.3
2022-04-05
Applied Intelligence
Abstract:As convolutional neural networks (CNNs) have become increasingly deeper in recent years, the requirements for the quantity of data and hardware resources have gradually increased. CNN also reveals salient redundancy in several tasks. The existing magnitude-based pruning methods are efficient, but the performance of the compressed network is unpredictable. While the accuracy loss after pruning based on the structure sensitivity is relatively slight, the process is time-consuming, and the algorithm complexity is notable. To fully combine the advantages of the two types of methods, we propose a novel automatic channel pruning method (ACP). Specifically, we first perform layerwise channel clustering via the similarity of the feature maps to perform preliminary pruning on the network. Then, a population initialization method is introduced to transform the pruned structure into a candidate population. Finally, we conduct iterative searching and optimization based on particle swarm optimization (PSO) to find the optimal compressed structure. The compact network is then retrained to mitigate the accuracy loss from pruning. Our method is evaluated against several state-of-the-art CNNs on three different classification datasets CIFAR-10/100 and ILSVRC-2012. On the ILSVRC-2012, when removing 64.36% parameters and 63.34% floating-point operations (FLOPs) of ResNet-50, the Top-1 and Top-5 accuracy drops are less than 0.9%. Moreover, we demonstrate that without harming the overall performance, it is possible to compress the SSD by more than 50% on the target detection dataset PASCAL VOC. This further verifies that the proposed method can also be applied to other CNNs and application scenarios.
computer science, artificial intelligence
What problem does this paper attempt to address?