Abstract:Dynamic channel pruning has been proved to be an effective method by dynamically adjusting the inference path to reduce the computing costs. However, in most existing work, the classification performance decreases rapidly with the increase of pruning rate because their pruning strategy weakens the representation ability of the model to a certain extent. To resolve this problem, a dynamic channel pruning method based on activation gate (DCPAG) is proposed, which can better maintain the classification performance while reducing the computing costs. First, a pipeline aiming for generating pruning strategy, namely channel pruning auxiliary (CPA) is proposed, which considers both the representation ability and computing costs. Second, the pruning strategy generated by CPA is embedded into dynamic rectifying linear unit (DyReLU) to form the embedded dynamic rectifying linear unit (EB-DyReLU), which achieves dynamic channel pruning while maintaining the representation capability. Third, each input sample was self-classified according to its identification difficulty during pruning, and additional training was given to hard samples to achieve better classification performance. Finally, some experiments are carried out on CIFAR-10 and ImageNet respectively to verify the effectiveness of DCPAG in accuracy and floating point of per second (FLOPs). The results show that the proposed method achieves better performance than other similar channel-based methods at the same pruning rate. Specifically, this method not only achieves 0.5-1.5% improvement in classification accuracy, but also reduces the computational costs by 5%-20%.

A Pruning Method Based on Conditional Misclassification

Pruning algorithm of classificaiton and regression decision tree based on Akaike information criteria

A Feature-map Discriminant Perspective for Pruning Deep Neural Networks

Class-Aware Pruning for Efficient Neural Networks

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Pruning Decision Tree Using Genetic Algorithms

An Analysis of Reduced Error Pruning

A New Pruning Method for Solving Decision Trees and Game Trees

An Approach to Improve Text Classification Efficiency.

The Conditioning Bias in Binary Decision Trees and Random Forests and Its Elimination

Handling Over-Fitting in Test Cost-Sensitive Decision Tree Learning by Feature Selection, Smoothing and Pruning

Classification Tree Pruning Under Covariate Shift

Pruning for Better Domain Generalizability

Dynamic channel pruning via activation gates

Data Pruning via Separability, Integrity, and Model Uncertainty-Aware Importance Sampling

Dynamical Channel Pruning by Conditional Accuracy Change for Deep Neural Networks

New Pruning Method Based on DenseNet Network for Image Classification

How Does Pruning Impact Long-Tailed Multi-Label Medical Image Classifiers?

Rethinking Class-Discrimination Based CNN Channel Pruning

On the Effectiveness of Machine Learning-based Call Graph Pruning: An Empirical Study