Abstract:Deep convolutional neural networks (CNN) have achieved remarkable performance with the cost of huge computation. As the CNN model becomes more complex and deeper, compressing CNN to sparse by pruning the redundant connection in networks has emerged as an attractive approach to reduce the amount of computation and memory requirement. In recent years, FPGAs have been demonstrated to be an effective hardware platform to accelerate CNN inference. However, most existing FPGA architectures focus on dense CNN models. The architecture designed for dense CNN models are inefficient when executing sparse models as most of the arithmetic operations involve addition and multiplication with zero operands. On the other hand, recent sparse FPGA accelerators only focus on FC layers. In this work, we aim to develop an FPGA accelerator for sparse CNNs. To efficiently deal with the irregular connection in the sparse convolutional layer, we propose a weight-oriented dataflow that processes each weight individually. Then we design an FPGA architecture which can handle input-weight connection and weight-output connection efficiently. For input-weight connection, we design a tile look-up table to eliminate the runtime indexing match of compressed weights. Moreover, we develop a weight layout to enable high on-chip memory access. To cooperate with the weight layout, a channel multiplexer is inserted to locate the address which can ensure no data access conflict. Experiments demonstrate that our accelerator can achieve 223.4-309.0 GOP/s for the modern CNNs on Xilinx ZCU102, which provides a 3.6x-12.9x speedup over previous dense CNN FPGA accelerators.

A Flexible Sparsity-Aware Accelerator with High Sensitivity and Efficient Operation for Convolutional Neural Networks

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

A Sparse Convolution Neural Network Accelerator with Bandwidth-Efficient Data Loopback Structure

An Efficient and Flexible Accelerator Design for Sparse Convolutional Neural Networks

SparseNN: A Performance-Efficient Accelerator for Large-Scale Sparse Neural Networks

A Reconfigurable Accelerator for Sparse Convolutional Neural Networks.

An Efficient Accelerator for Sparse Convolutional Neural Networks

A Convolutional Spiking Neural Network Accelerator with the Sparsity-Aware Memory and Compressed Weights

An Efficient Accelerator for Multiple Convolutions From the Sparsity Perspective

An Efficient Sparse CNNs Accelerator on FPGA

Accelerator for Sparse Convolutional Neural Networks Based on Shift Units

Work-in-Progress: A High-performance FPGA Accelerator for Sparse Neural Networks

An Efficient CNN Accelerator for Pattern-Compressed Sparse Neural Networks on FPGA

Design of Sparse Convolutional Neural Network Accelerator

An Efficient Hardware Accelerator for Sparse Convolutional Neural Networks on FPGAs

A Low-Power Sparse Convolutional Neural Network Accelerator with Pre-Encoding Radix-4 Booth Multiplier

A High-performance Inference Accelerator Exploiting Patterned Sparsity in CNNs

An Energy-Efficient Spiking Neural Network Accelerator Based on Spatio-Temporal Redundancy Reduction

VSCNN: Convolution Neural Network Accelerator With Vector Sparsity

A Precision-Scalable Deep Neural Network Accelerator with Activation Sparsity Exploitation

FPGA Accelerator for CNN: an Exploration of the Kernel Structured Sparsity and Hybrid Arithmetic Computation