Abstract:Over the past few years, 2-D convolutional neural networks (CNNs) have demonstrated their great success in a wide range of 2-D computer vision applications, such as image classification and object detection. At the same time, 3-D CNNs, as a variant of 2-D CNNs, have shown their excellent ability to analyze 3-D data, such as video and geometric data. However, the heavy algorithmic complexity of 2-D and 3-D CNNs imposes a substantial overhead over the speed of these networks, which limits their deployment in real-life applications. Although various domain-specific accelerators have been proposed to address this challenge, most of them only focus on accelerating 2-D CNNs, without considering their computational efficiency on 3-D CNNs. In this article, we propose a unified hardware architecture to accelerate both 2-D and 3-D CNNs with high hardware efficiency. Our experiments demonstrate that the proposed accelerator can achieve up to 92.4% and 85.2% multiply-accumulate efficiency on 2-D and 3-D CNNs, respectively. To improve the hardware performance, we propose a hardware-friendly quantization approach called static block floating point (BFP), which eliminates the frequent representation conversions required in traditional dynamic BFP arithmetic. Comparing with the integer linear quantization using zero-point, the static BFP quantization can decrease the logic resource consumption of the convolutional kernel design by nearly 50% on a field-programmable gate array (FPGA). Without time-consuming retraining, the proposed static BFP quantization is able to quantize the precision to 8-bit mantissa with negligible accuracy loss. As different CNNs on our reconfigurable system require different hardware and software parameters to achieve optimal hardware performance and accuracy, we also propose an automatic tool for parameter optimization. Based on our hardware design and optimization, we demonstrate that the proposed accelerator can achieve 3.8-5.6 times higher energy efficiency than graphics processing unit (GPU) implementation. Comparing with the state-of-the-art FPGA-based accelerators, our design achieves higher generality and up to 1.4-2.2 times higher resource efficiency on both 2-D and 3-D CNNs.

Improving CNN Performance on FPGA Clusters Through Topology Exploration

Energy-Efficient Cnn Implementation on A Deeply Pipelined Fpga Cluster

Improving HW/SW Adaptability for Accelerating CNNs on FPGAs Through A Dynamic/Static Co-Reconfiguration Approach

Optimization of Convolution Neural Network Algorithm Based on FPGA

Communication-Aware and Resource-Efficient NoC-Based Architecture for CNN Acceleration

Throughput-Optimized FPGA Accelerator for Deep Convolutional Neural Networks

Improving the Performance of OpenCL-based FPGA Accelerator for Convolutional Neural Network

FPGA-Based High-Throughput CNN Hardware Accelerator With High Computing Resource Utilization Ratio

A Holistic Approach for Optimizing DSP Block Utilization of a CNN implementation on FPGA

Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs.

A Scalable FPGA Accelerator for Convolutional Neural Networks.

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

A Data-Center FPGA Acceleration Platform for Convolutional Neural Networks

A Power-Efficient and High Performance FPGA Accelerator for Convolutional Neural Networks: Work-in-progress.

An Efficient Sparse CNNs Accelerator on FPGA

XFER: A Novel Design to Achieve Super-Linear Performance on Multiple FPGAs for Real-Time AI.

Heterogeneous FPGA-Based Cost-Optimal Design for Timing-Constrained CNNs

High-Performance Acceleration of 2-D and 3-D CNNs on FPGAs Using Static Block Floating Point

An FPGA-Based CNN Accelerator Integrating Depthwise Separable Convolution

Throughput-Optimized Opencl-Based Fpga Accelerator For Large-Scale Convolutional Neural Networks

A Grain-Adaptive Computing Structure for FPGA CNN Acceleration.