Abstract:Over the past few years, 2-D convolutional neural networks (CNNs) have demonstrated their great success in a wide range of 2-D computer vision applications, such as image classification and object detection. At the same time, 3-D CNNs, as a variant of 2-D CNNs, have shown their excellent ability to analyze 3-D data, such as video and geometric data. However, the heavy algorithmic complexity of 2-D and 3-D CNNs imposes a substantial overhead over the speed of these networks, which limits their deployment in real-life applications. Although various domain-specific accelerators have been proposed to address this challenge, most of them only focus on accelerating 2-D CNNs, without considering their computational efficiency on 3-D CNNs. In this article, we propose a unified hardware architecture to accelerate both 2-D and 3-D CNNs with high hardware efficiency. Our experiments demonstrate that the proposed accelerator can achieve up to 92.4% and 85.2% multiply-accumulate efficiency on 2-D and 3-D CNNs, respectively. To improve the hardware performance, we propose a hardware-friendly quantization approach called static block floating point (BFP), which eliminates the frequent representation conversions required in traditional dynamic BFP arithmetic. Comparing with the integer linear quantization using zero-point, the static BFP quantization can decrease the logic resource consumption of the convolutional kernel design by nearly 50% on a field-programmable gate array (FPGA). Without time-consuming retraining, the proposed static BFP quantization is able to quantize the precision to 8-bit mantissa with negligible accuracy loss. As different CNNs on our reconfigurable system require different hardware and software parameters to achieve optimal hardware performance and accuracy, we also propose an automatic tool for parameter optimization. Based on our hardware design and optimization, we demonstrate that the proposed accelerator can achieve 3.8-5.6 times higher energy efficiency than graphics processing unit (GPU) implementation. Comparing with the state-of-the-art FPGA-based accelerators, our design achieves higher generality and up to 1.4-2.2 times higher resource efficiency on both 2-D and 3-D CNNs.

Accelerating Block-Matching and 3D Filtering-Based Image Denoising Algorithm on FPGAs

High-Performance Acceleration of 2-D and 3-D CNNs on FPGAs Using Static Block Floating Point

An OpenCLTM Implementation of WebP Accelerator on FPGAs.

FEB3D - An Efficient FPGA-Accelerated Compression Framework for Microscopy Images.

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

A High-Performance Accelerator for Real-Time Super-Resolution on Edge FPGAs

Towards a Uniform Architecture for the Efficient Implementation of 2D and 3D Deconvolutional Neural Networks on FPGAs

An Image Histogram Equalization Acceleration Method for Field-Programmable Gate Arrays Based on a Two-Dimensional Configurable Pipeline

Accelerating convolutional neural networks on FPGAs

Design and Implement of Median Filter Toward Remote Sensing Images Based on FPGA

A Ubiquitous Machine Learning Accelerator With Automatic Parallelization on FPGA

An OpenCL-Based FPGA Accelerator for Faster R-CNN

Image Denoising Method Based on FPGA in Digital Video Image Transmission System

FD-CNN: A Frequency-Domain FPGA Acceleration Scheme for CNN-Based Image-Processing Applications

A Block-Floating-Point Arithmetic Based FPGA Accelerator for Convolutional Neural Networks

Acceleration of 3D feature-enhancing noise filtering in hybrid CPU/GPU systems

Blind Image Denoising Based on Noise Level Estimation

New noise reduction method based on FPGA for a stereoscopic video system

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

Real-time hardware accelerator for single image haze removal using dark channel prior and guided filter

Real-time FPGA Implementation of an Optic Flow Algorithm Suitable for Background Oriented Schlieren Technique