Abstract:The rapidly growing parameter volume of deep neural networks (DNNs) hinders the artificial intelligence applications on resource constrained devices, such as mobile and wearable devices. Neural network pruning, as one of the mainstream model compression techniques, is under extensive study to reduce the model size and thus the amount of computation. And thereby, the state-of-the-art DNNs are able to be deployed on those devices with high runtime energy efficiency. In contrast to irregular pruning that incurs high index storage and decoding overhead, structured pruning techniques have been proposed as the promising solutions. However, prior studies on structured pruning tackle the problem mainly from the perspective of facilitating hardware implementation, without diving into the deep to analyze the characteristics of sparse neural networks. The neglect on the study of sparse neural networks causes inefficient trade-off between regularity and pruning ratio. Consequently, the potential of structurally pruning neural networks is not sufficiently mined.In this work, we examine the structural characteristics of the irregularly pruned weight matrices, such as the diverse redundancy of different rows, the sensitivity of different rows to pruning, and the position characteristics of retained weights. By leveraging the gained insights as a guidance, we first propose the novel block-max weight masking (BMWM) method, which can effectively retain the salient weights while imposing high regularity to the weight matrix. As a further optimization, we propose a density-adaptive regular-block (DARB) pruning that can effectively take advantage of the intrinsic characteristics of neural networks, and thereby outperform prior structured pruning work with high pruning ratio and decoding efficiency. Our experimental results show that DARB can achieve 13× to 25× pruning ratio, which are 2.8× to 4.3× improvements than the state-of-the-art counterparts on multiple neural network models and tasks. Moreover, DARB can achieve 14.3× decoding efficiency than block pruning with higher pruning ratio.

SMVAR: A Novel RNN Accelerator Based on Non-blocking Data Distribution Structure

SUBP: Soft Uniform Block Pruning for 1 X N Sparse CNNs Multithreading Acceleration

SUBP: Soft Uniform Block Pruning for 1xn Sparse CNNs Multithreading Acceleration

BLCR: Towards Real-time DNN Execution with Block-based Reweighted Pruning

MASR: A Modular Accelerator for Sparse RNNs

Learning the sparsity for ReRAM - mapping and pruning sparse neural network for ReRAM based accelerator.

Block-Sparse Recurrent Neural Networks

SoBS-X: Squeeze-Out Bit Sparsity for ReRAM-Crossbar-Based Neural Network Accelerator.

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

A Flexible Yet Efficient DNN Pruning Approach for Crossbar-Based Processing-in-Memory Architectures.

GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity

A Universal RRAM-Based DNN Accelerator with Programmable Crossbars Beyond MVM Operator

Structured Pruning of Recurrent Neural Networks through Neuron Selection

SNrram: an Efficient Sparse Neural Network Computation Architecture Based on Resistive Random-Access Memory.

APQ: Automated DNN Pruning and Quantization for ReRAM-Based Accelerators

AUTO-PRUNE

HBP: Hierarchically Balanced Pruning and Accelerator Co-Design for Efficient DNN Inference.

A Fine-Grained Sparse Accelerator for Multi-Precision DNN.

DARB: A Density-Adaptive Regular-Block Pruning for Deep Neural Networks.

Efficient and Effective Sparse LSTM on FPGA with Bank-Balanced Sparsity

DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks