Filter Pruning with a Feature Map Entropy Importance Criterion for Convolution Neural Networks Compressing
Jielei Wang,Ting Jiang,Zongyong Cui,Zongjie Cao
DOI: https://doi.org/10.1016/j.neucom.2021.07.034
IF: 6
2021-01-01
Neurocomputing
Abstract:Deep Neural Networks (DNN) has made significant progress in recent years. However, its high computing and storage costs make it challenging to apply on resource-limited platforms or edge computation scenarios. Recent studies have shown that model pruning is an effective method to solve this problem. Typically, the model pruning method is a three-stage pipeline: training, pruning, and fine-tuning. In this work, a novel structured pruning method for Convolutional Neural Networks (CNN) compression is proposed, where filter-level redundant weights are pruned according to entropy importance criteria (termed FPEI). In short, the FPEI criterion, which works in the stage of pruning, defines the importance of the filter according to the entropy of feature maps. If a feature map contains very little information, it should not contribute much to the whole network. By removing these uninformative feature maps, their corresponding filters in the current layer and kernels in the next layer can be removed simultaneously. Consequently, the computing and storage costs are significantly reduced. Moreover, because our method cannot show the advantages of the existing ResNet pruning strategy, we propose a dimensionality reduction (DR) pruning strategy for ResNet structured networks. Experiments on several datasets demonstrate that our method is effective. In the experiment about the VGG-16 model on the SVHN dataset, we removed 91.31% of the parameters, from 14.73M to 1.28M, achieving a 63.77% reduction in the FLOPs, from 313.4M to 113.5M, and 1.73 times speedups of model inference.