Abstract:Neural Networks (NNs) have made a significant breakthrough in many fields, while they also pose a great challenge to hardware platforms since the state-of-the-art neural networks are both communicational- and computational-intensive. Researchers proposed model compression algorithms using sparsification and quantization, along with specific hardware architecture designs, to accelerate various applications. However, the irregularity of memory access caused by the sparsity severely damages the regularity of intensive computation loops. Therefore, the architecture design for sparse neural networks is crucial to better software and hardware co-design for neural network applications. To face these challenges, this paper first analyzes the computation patterns of different NN structures and unify them into the form of sparse matrix-vector multiplication, sparse matrix-matrix multiplication, and element-wise multiplication. On the basis of the EIE which supports only the fully-connected network and recurrent neural network (RNN), we expand it to support the convolution neural network (CNN) using the input vector transform unit. This paper designs a multi-precision multiplier with supporting datapath, which makes the proposed architecture have a better acceleration effect in the low-bit quantization with the same hardware architecture. The proposed accelerator architecture can achieve the equivalent performance and energy efficiency up to 574.2 GOPS, 42.8 GOPS/W for CNN and 110.4 GOPS, 8.24 GOPS/W for RNN under 4-bit quantization on Xilinx XCKU115 FPGA running at 200MHz. And it is the state-of-the-art accelerator supporting CNN-RNN-based models like the long-term recurrent convolutional network with 571.1 GOPS performance and 42.6 GOPS/W energy efficiency under 4-bit data format.

An Energy-Efficient Accelerator for Hybrid Bit-Width DNNs

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

An Energy-Efficient Mixed-Bitwidth Systolic Accelerator for NAS-Optimized Deep Neural Networks

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

HybridDNN: A Framework for High-Performance Hybrid DNN Accelerator Design and Implementation.

A Fine-Grained Sparse Accelerator for Multi-Precision DNN.

FAMES: Fast Approximate Multiplier Substitution for Mixed-Precision Quantized DNNs--Down to 2 Bits!

Leveraging Bit-Serial Architectures for Hardware-Oriented Deep Learning Accelerators with Column-Buffering Dataflow

An FPGA-Based Reconfigurable Accelerator for Low-Bit DNN Training

EERA-DNN: an Energy-Efficient Reconfigurable Architecture for DNNs with Hybrid Bit-Width and Logarithmic Multiplier.

A Reconfigurable DNN Training Accelerator on FPGA

A Precision-Scalable Deep Neural Network Accelerator with Activation Sparsity Exploitation

Design Flow of Accelerating Hybrid Extremely Low Bit-Width Neural Network in Embedded FPGA

A High Performance Multi-Bit-Width Booth Vector Systolic Accelerator for NAS Optimized Deep Learning Neural Networks

Exploiting Bit Sparsity in Both Activation and Weight in Neural Networks Accelerators

Optimizing FPGA-Based DNN Accelerator with Shared Exponential Floating-Point Format

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

Elastic Significant Bit Quantization and Acceleration for Deep Neural Networks

An All-Digital Compute-In-Memory FPGA Architecture for Deep Learning Acceleration

Exploiting Dynamic Bit Sparsity in Activation for Deep Neural Network Acceleration.

A Low-Latency DNN Accelerator Enabled by DFT-Based Convolution Execution Within Crossbar Arrays