Abstract:Field-programmable gate array (FPGA)-based accelerators for convolutional neural network (CNN) inference have received significant attention in recent years. The reported designs tend to adopt a similar underlying approach based on multiplier-accumulator (MAC) arrays, which yields strong demand for the available on-chip DSP blocks, while leaving FPGA logic and memory resources underutilized. The practical outcome is that the computational roof of the accelerator is bound by the number of DSP blocks offered by the target FPGA. In addition, integrating the CNN accelerator with other functional units that may also need DSP blocks would degrade the inference performance. Leveraging the robustness of inference accuracy to limited arithmetic precision, we propose a transformation to the convolution computation, which leads to transformation of the accelerator design space and relaxes the pressure on the required DSP resources. Through analytical and empirical evaluations, we demonstrate that our approach enables us to strike a favorable balance between utilization of the FPGA on-chip memory, logic, and DSP resources, due to which, our accelerator considerably outperforms state of the art. We report the effectiveness of our approach on a variety of FPGA devices, including Cyclone-V, Stratix-V, and Arria-10, which are used in large number of applications, ranging from embedded settings to high performance computing. Our proposed technique yields 1.5x throughput improvement and 4x DSP resource reduction compared to the best frequency domain convolution-based accelerator, and 2.5x boost in raw arithmetic performance and 8.4x saving in DSPs compared to a state-of-the-art sparse convolution-based accelerator.

A Hardware Accelerator for Standard Convolution and Depthwise Convolution

DSP-Efficient Hardware Acceleration of Convolutional Neural Network Inference on FPGAs

A High-performance Hardware Accelerator Using a Fusion Approach of Convolution and Pooling

An FPGA-Based CNN Accelerator Integrating Depthwise Separable Convolution

An Efficient FPGA-based Depthwise Separable Convolutional Neural Network Accelerator with Hardware Pruning.

A High Performance Reconfigurable Hardware Architecture for Lightweight Convolutional Neural Network

Toward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs

FPGA-based Accelerator for Convolution Operations

FPGA deep learning acceleration based on convolutional neural network

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

A High-Performance Accelerator for Large-Scale Convolutional Neural Networks

A FPGA-based Hardware Accelerator for Multiple Convolutional Neural Networks

WPU: A FPGA-based Scalable, Efficient and Software/Hardware Co-design Deep Neural Network Inference Acceleration Processor

An Efficient Convolutional Neural Network Accelerator on FPGA

FPGA-based Accelerator for Convolutional Neural Network

High Throughput CNN Accelerator Design Based on FPGA

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

FPGA-Based High-Throughput CNN Hardware Accelerator With High Computing Resource Utilization Ratio

A High Energy Efficiency and Low Resource Consumption FPGA Accelerator for Convolutional Neural Network

Hardware-Efficient Template-Based Deep CNNs Accelerator Design

An FPGA-based HEVC post-processing CNN hardware accelerator