Abstract:Deep convolutional neural network (DNN) has demonstrated phenomenal success and been widely used in many computer vision tasks. However, its enormous model size and high computing complexity prohibits its wide deployment into resource limited embedded system, such as FPGA and mGPU. As the two most widely adopted model compression techniques, weight pruning and quantization compress DNN model through introducing weight sparsity (i.e., forcing partial weights as zeros) and quantizing weights into limited bit-width values, respectively. Although there are works attempting to combine the weight pruning and quantization, we still observe disharmony between weight pruning and quantization, especially when more aggressive compression schemes (e.g., Structured pruning and low bit-width quantization) are used. In this work, taking FPGA as the test computing platform and Processing Elements (PE) as the basic parallel computing unit, we first propose a PE-wise structured pruning scheme, which introduces weight sparsification with considering of the architecture of PE. In addition, we integrate it with an optimized weight ternarization approach which quantizes weights into ternary values ({-1,0,+1}), thus converting the dominant convolution operations in DNN from multiplication-and-accumulation (MAC) to addition-only, as well as compressing the original model (from 32-bit floating point to 2-bit ternary representation) by at least 16 times. Then, we investigate and solve the coexistence issue between PE-wise Structured pruning and ternarization, through proposing a Weight Penalty Clipping (WPC) technique with self-adapting threshold. Our experiment shows that the fusion of our proposed techniques can achieve the best state-of-the-art ∼21× PE-wise structured compression rate with merely 1.74%/0.94% (top-1/top-5) accuracy degradation of ResNet-18 on ImageNet dataset.

Tight Compression: Compressing CNN Model Tightly Through Unstructured Pruning and Simulated Annealing Based Permutation

Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation

Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Structured Deep Neural Network Pruning by Varying Regularization Parameters.

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Data Stream Oriented Fine-grained Sparse CNN Accelerator with Efficient Unstructured Pruning Strategy

Efficient Network Compression Through Smooth-Lasso Constraint

Structured Term Pruning for Computational Efficient Neural Networks Inference

AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio.

Multi-grained Pruning Method of Convolutional Neural Network.

PCNN: Pattern-based Fine-Grained Regular Pruning Towards Optimizing CNN Accelerators

Harmonious Coexistence of Structured Weight Pruning and Ternarization for Deep Neural Networks

Structured Compression by Weight Encryption for Unstructured Pruning and Quantization

An efficient pruning and fine-tuning method for deep spiking neural network

LAPP: Layer Adaptive Progressive Pruning for Compressing CNNs from Scratch

StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs

Learning Low Resource Consumption CNN through Pruning and Quantization