Abstract:Deep neural networks (DNNs) have demonstrated remarkable success in various fields. However, the large number of floating-point operations (FLOPs) in DNNs poses challenges for their deployment in resource-constrained applications, e.g., edge devices. To address the problem, pruning has been introduced to reduce the computational cost in executing DNNs. Previous pruning strategies are based on weight values, gradient values and activation outputs. Different from previous pruning solutions, in this paper, we propose a class-aware pruning technique to compress DNNs, which provides a novel perspective to reduce the computational cost of DNNs. In each iteration, the neural network training is modified to facilitate the class-aware pruning. Afterwards, the importance of filters with respect to the number of classes is evaluated. The filters that are only important for a few number of classes are removed. The neural network is then retrained to compensate for the incurred accuracy loss. The pruning iterations end until no filter can be removed anymore, indicating that the remaining filters are very important for many classes. This pruning technique outperforms previous pruning solutions in terms of accuracy, pruning ratio and the reduction of FLOPs. Experimental results confirm that this class-aware pruning technique can significantly reduce the number of weights and FLOPs, while maintaining a high inference accuracy. Our code is available at https://github.com/HWAI-TUDa/Class-Aware-Pruning

VeriPrune: Equivalence verification of node pruned neural network

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Class-Aware Pruning for Efficient Neural Networks

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Pruning and Slicing Neural Networks using Formal Verification

A Single-sample Pruning and Clustering Method for Neural Network Verification

Expediting Neural Network Verification via Network Reduction

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Shapley Pruning for Neural Network Compression

Exploring The Neural Burden In Pruned Models: An Insight Inspired By Neuroscience

Novel Pruning of Dendritic Neuron Models for Improved System Implementation and Performance

Pruning and quantization for deep neural network acceleration: A survey

Differential Evolution Based Layer-Wise Weight Pruning for Compressing Deep Neural Networks

Pruning the Deep Neural Network by Similar Function

Network Automatic Pruning: Start NAP and Take a Nap

Quantisation and Pruning for Neural Network Compression and Regularisation

Pruning at a Glance: Global Neural Pruning for Model Compression

Overview of Deep Convolutional Neural Network Pruning

Multiple Sparse Spaces Network Pruning Via a Joint Similarity Criterion

A Dynamic Pruning Method on Multiple Sparse Structures in Deep Neural Networks

Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures