Towards Efficient Filter Pruning Via Adaptive Automatic Structure Search
Xiaozhou Xu,Jun Chen,Zhishan Li,Hongye Su,Lei Xie
DOI: https://doi.org/10.1016/j.engappai.2024.108398
IF: 8
2024-01-01
Engineering Applications of Artificial Intelligence
Abstract:Filter pruning is a critical technique for compressing large convolutional neural networks, making it possible to deploy deep networks on resource-limited edge devices. However, previous pruning methods typically concentrate on removing filters with rule-of-thumb designs and empirically set the pruning rate per layer, which is prone to produce sub-optimal pruning. To address this issue, we develop a Filter Pruning method via Adaptive Automatic Structure Search (FP-AASS), which treats filter pruning as a structure optimization task. In FP-AASS, we employ the artificial bee colony algorithm to automatically search for the optimal pruned structure that meets the FLOPs and parameters constraints. The structure search process is divided into two stages through an additional adjustment phase to reduce the time consumption caused by the large search space. We also adopt adaptive batch normalization in the evaluation process to accelerate the search process and reinforce the relevance between the evaluation accuracy and the final converged accuracy achieved by the pruning candidates. Through extensive experiments, FP-AASS has been demonstrated to be an effective automatic filter pruning method. For instance, FP-AASS slims a standard ResNet-50 to a narrower one with 56.8% FLOPs and 55.0% parameters reduction with only a 1.50% decrease in top-1 accuracy on ImageNet-1K. In contrast, under similar FLOPs pruning rates, other representative methods show a top-1 accuracy decrease of at least 1.70% or more.