Abstract:Convolutional neural networks (CNNs) have been widely used in image classification and recognition due to their effectiveness; however, CNNs use a large volume of weight data that is difficult to store in on-chip memory of embedded designs. Pruning can compress the CNN model at a small accuracy loss; however, a pruned CNN model operates slower when implemented on a parallel architecture. In this paper, a hardware-oriented CNN compression strategy is proposed; a deep neural network (DNN) model is divided into "no-pruning layers ( $NP$ -layers)" and "pruning layers ( $P$ -layers)". A $NP$ -layer has a regular weights distribution for parallel computing and high performance. A $P$ -layer is irregular due to pruning, but it generates a high compression ratio. Uniform and incremental quantization schemes are used to achieve a tradeoff between compression ratio and processing efficiency at a small loss in accuracy. A distributed convolutional architecture with several parallel finite impulse response (FIR) filters is further proposed for the regular model in the $NP$ -layers. A shift-accumulator based processing element with an activation-driven data flow (ADF) is proposed for the irregular sparse model in the $P$ -layers. Based on the proposed compression strategy and hardware architecture, a hardware/algorithm co-optimization (HACO) approach is proposed for implementing a $NP-P$ hybrid compressed CNN model on FPGAs. For a hardware accelerator on a single FPGA chip without the use of off-chip memory, a $27.5times $ compression ratio is achieved with 0.44% top-5 accuracy loss for VGG-16. The implementation of the compressed VGG-16 model on a Xilinx VCU118 evaluation board processes 83.0 frames per second (FPS) for image applications, this is $1.8times $ superior than the state-of-the-art design found in the technical literature.

Deep Neural Network Interlayer Feature Map Compression Based on Least-Squares Fitting.

MLCNN: Cross-Layer Cooperative Optimization and Accelerator Architecture for Speeding Up Deep Learning Applications

Deep Neural Network Acceleration with Sparse Prediction Layers

Memory-Efficient Compression Based on Least-Squares Fitting in Convolutional Neural Network Accelerators.

Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

An Efficient CNN Inference Accelerator Based on Intra- and Inter-Channel Feature Map Compression

ASC: Adaptive Scale Feature Map Compression for Deep Neural Network

Accelerating Deep Neural Networks by Combining Block-Circulant Matrices and Low-Precision Weights

A Computing Efficient Hardware Architecture for Sparse Deep Neural Network Computing

Efficient Network Compression Through Smooth-Lasso Constraint

Relative Indexed Compressed Sparse Filter Encoding Format for Hardware-Oriented Acceleration of Deep Convolutional Neural Networks

Iterative Deep Model Compression and Acceleration in the Frequency Domain.

Improved JPEG Lossless Compression for Compression of Intermediate Layers in Neural Networks Based on Compute-In-Memory

High Performance CNN Accelerators Based on Hardware and Algorithm Co-Optimization

FPGA-Based High-Performance Data Compression Deep Neural Network Accelerator

Intermediate Deep Feature Compression: the Next Battlefield of Intelligent Sensing

Convolutional neural network model compression method

FSNet: Compression of Deep Convolutional Neural Networks by Filter Summary

Toward Intelligent Sensing: Intermediate Deep Feature Compression

Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding

A Survey of Model Compression and Acceleration for Deep Neural Networks.