Abstract:Over the past few years, 2-D convolutional neural networks (CNNs) have demonstrated their great success in a wide range of 2-D computer vision applications, such as image classification and object detection. At the same time, 3-D CNNs, as a variant of 2-D CNNs, have shown their excellent ability to analyze 3-D data, such as video and geometric data. However, the heavy algorithmic complexity of 2-D and 3-D CNNs imposes a substantial overhead over the speed of these networks, which limits their deployment in real-life applications. Although various domain-specific accelerators have been proposed to address this challenge, most of them only focus on accelerating 2-D CNNs, without considering their computational efficiency on 3-D CNNs. In this article, we propose a unified hardware architecture to accelerate both 2-D and 3-D CNNs with high hardware efficiency. Our experiments demonstrate that the proposed accelerator can achieve up to 92.4% and 85.2% multiply-accumulate efficiency on 2-D and 3-D CNNs, respectively. To improve the hardware performance, we propose a hardware-friendly quantization approach called static block floating point (BFP), which eliminates the frequent representation conversions required in traditional dynamic BFP arithmetic. Comparing with the integer linear quantization using zero-point, the static BFP quantization can decrease the logic resource consumption of the convolutional kernel design by nearly 50% on a field-programmable gate array (FPGA). Without time-consuming retraining, the proposed static BFP quantization is able to quantize the precision to 8-bit mantissa with negligible accuracy loss. As different CNNs on our reconfigurable system require different hardware and software parameters to achieve optimal hardware performance and accuracy, we also propose an automatic tool for parameter optimization. Based on our hardware design and optimization, we demonstrate that the proposed accelerator can achieve 3.8-5.6 times higher energy efficiency than graphics processing unit (GPU) implementation. Comparing with the state-of-the-art FPGA-based accelerators, our design achieves higher generality and up to 1.4-2.2 times higher resource efficiency on both 2-D and 3-D CNNs.

BOOST: Block Minifloat-Based On-Device CNN Training Accelerator with Transfer Learning

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support

A Block-Floating-Point Arithmetic Based FPGA Accelerator for Convolutional Neural Networks

FlexBCM: Hybrid Block-Circulant Neural Network and Accelerator Co-Search on FPGAs

HBCA: A Toolchain for High-Accuracy Branch-Fused CNN Accelerator on FPGA with Dual-Decimal-Fused Technique

High-Performance Acceleration of 2-D and 3-D CNNs on FPGAs Using Static Block Floating Point

A Parallel Loading Based Accelerator for Convolution Neural Network

Low Precision Floating Point Arithmetic for High Performance FPGA-based CNN Acceleration

An Efficient Streaming Accelerator for Low Bit-Width Convolutional Neural Networks

High Performance CNN Accelerators Based on Hardware and Algorithm Co-Optimization

Compressed CNN Training with FPGA-based Accelerator

A FPGA-based end-to-end acceleration framework for fast deployment of Convolutional Neural Networks

Optimizing CNN Hardware Acceleration with Configurable Vector Units and Feature Layout Strategies

BISWSRBS: A Winograd-based CNN Accelerator with a Fine-grained Regular Sparsity Pattern and Mixed Precision Quantization

A High Performance Multi-Bit-Width Booth Vector Systolic Accelerator for NAS Optimized Deep Learning Neural Networks

A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks

Acceleration of Deep Neural Network Training Using Field Programmable Gate Arrays

Hardware-Efficient Template-Based Deep CNNs Accelerator Design

Toward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs