Abstract:Neural Networks (NNs) have made a significant breakthrough in many fields, while they also pose a great challenge to hardware platforms since the state-of-the-art neural networks are both communicational- and computational-intensive. Researchers proposed model compression algorithms using sparsification and quantization, along with specific hardware architecture designs, to accelerate various applications. However, the irregularity of memory access caused by the sparsity severely damages the regularity of intensive computation loops. Therefore, the architecture design for sparse neural networks is crucial to better software and hardware co-design for neural network applications. To face these challenges, this paper first analyzes the computation patterns of different NN structures and unify them into the form of sparse matrix-vector multiplication, sparse matrix-matrix multiplication, and element-wise multiplication. On the basis of the EIE which supports only the fully-connected network and recurrent neural network (RNN), we expand it to support the convolution neural network (CNN) using the input vector transform unit. This paper designs a multi-precision multiplier with supporting datapath, which makes the proposed architecture have a better acceleration effect in the low-bit quantization with the same hardware architecture. The proposed accelerator architecture can achieve the equivalent performance and energy efficiency up to 574.2 GOPS, 42.8 GOPS/W for CNN and 110.4 GOPS, 8.24 GOPS/W for RNN under 4-bit quantization on Xilinx XCKU115 FPGA running at 200MHz. And it is the state-of-the-art accelerator supporting CNN-RNN-based models like the long-term recurrent convolutional network with 571.1 GOPS performance and 42.6 GOPS/W energy efficiency under 4-bit data format.

A Precision-Scalable Deep Neural Network Accelerator with Activation Sparsity Exploitation

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Exploiting Dynamic Bit Sparsity in Activation for Deep Neural Network Acceleration.

Exploiting Bit Sparsity in Both Activation and Weight in Neural Networks Accelerators

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

A Fine-Grained Sparse Accelerator for Multi-Precision DNN.

SparseNN: an Energy-Efficient Neural Network Accelerator Exploiting Input and Output Sparsity

SPARK: Scalable and Precision-Aware Acceleration of Neural Networks Via Efficient Encoding

Low-Complexity Precision-Scalable Multiply-Accumulate Unit Architectures for Deep Neural Network Accelerators

A Flexible Sparsity-Aware Accelerator with High Sensitivity and Efficient Operation for Convolutional Neural Networks

AdaS: A Fast and Energy-Efficient CNN Accelerator Exploiting Bit-Sparsity

A Low-Power Sparse Convolutional Neural Network Accelerator with Pre-Encoding Radix-4 Booth Multiplier

SPAT: FPGA-based Sparsity-Optimized Spiking Neural Network Training Accelerator with Temporal Parallel Dataflow

A Low-Power Accelerator for Deep Neural Networks with Enlarged Near-Zero Sparsity

A Precision-Scalable Energy-Efficient Convolutional Neural Network Accelerator.

SparseNN: A Performance-Efficient Accelerator for Large-Scale Sparse Neural Networks

FuseKNA: Fused Kernel Convolution based Accelerator for Deep Neural Networks

Accelerator for Sparse Convolutional Neural Networks Based on Shift Units

An Efficient Accelerator for Sparse Convolutional Neural Networks

An Energy-Efficient Spiking Neural Network Accelerator Based on Spatio-Temporal Redundancy Reduction

An Efficient Hardware Architecture for DNN Training by Exploiting Triple Sparsity