Abstract:Deep convolutional neural networks (CNNs) have achieved tremendous successes but tend to suffer from high computation costs mainly due to heavy over-parameterization, resulting in the difficulty of directly applying them to the ever-growing application demands based on low-end edge devices with strong power restriction and real-time inference requirement. Recently, there has much research attention devoted to compressing the network via pruning to address this issue. Most of the existing methods rely on some hand-designed pruning rules, which suffer from several limitations. Firstly, manually designed rules are only applicable to limited application scenarios, which can hardly generalize well in a broader scope. And these rules are typically designed based on human experience and via trial and error, and thus highly subjective. Then, channels of different layers in a network may have diverse distributions, which means the same pruning rule is not appropriate for each layer. To address these limitations, we propose a novel channel pruning scheme, in which the task-irrelevant channels are removed in a task-driven manner. Specifically, an adaptively differentiable search module is proposed to find the best pruning rule automatically for different layers in CNNs under sparsity constraints. Besides, we employed knowledge distillation to alleviate the excessive performance loss. Once the training process is finished, a compact network will be obtained by removing channels based on layer-wise pruning rules. We have evaluated the proposed method on some well-known benchmark datasets including CIFAR, MNIST, and ImageNet in comparison to several state-of-the-art pruning methods. Experimental results demonstrate the superiority of our method over the compared ones in terms of both parameters and FLOPs reduction.

A Channel-level Pruning Strategy for Convolutional Layers in CNNs

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

CCPrune: Collaborative Channel Pruning for Learning Compact Convolutional Networks

Structured Deep Neural Network Pruning by Varying Regularization Parameters.

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Loss-Driven Channel Pruning of Convolutional Neural Networks.

Channel Pruning Based on Mean Gradient for Accelerating Convolutional Neural Networks

An Efficient Channel-level Pruning for CNNs without Fine-tuning

Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

Convolutional Neural Network Compression Method Based on Multi-Factor Channel Pruning

Structure Characteristic-Aware Pruning Strategy for Convolutional Neural Networks.

Efficient Network Compression Through Smooth-Lasso Constraint

Learning Low Resource Consumption CNN through Pruning and Quantization

A Sensitivity-based Pruning Method for Convolutional Neural Networks.

An Automatically Layer-wise Searching Strategy for Channel Pruning Based on Task-driven Sparsity Optimization

Structure Level Pruning of Efficient Convolutional Neural Networks with Sparse Group LASSO

Channel Pruning for Efficient Convolution Neural Networks

Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networks

Network Pruning Using Sparse Learning and Genetic Algorithm

Channel Pruning Guided by Global Channel Relation.