Abstract:Over the past few years, 2-D convolutional neural networks (CNNs) have demonstrated their great success in a wide range of 2-D computer vision applications, such as image classification and object detection. At the same time, 3-D CNNs, as a variant of 2-D CNNs, have shown their excellent ability to analyze 3-D data, such as video and geometric data. However, the heavy algorithmic complexity of 2-D and 3-D CNNs imposes a substantial overhead over the speed of these networks, which limits their deployment in real-life applications. Although various domain-specific accelerators have been proposed to address this challenge, most of them only focus on accelerating 2-D CNNs, without considering their computational efficiency on 3-D CNNs. In this article, we propose a unified hardware architecture to accelerate both 2-D and 3-D CNNs with high hardware efficiency. Our experiments demonstrate that the proposed accelerator can achieve up to 92.4% and 85.2% multiply-accumulate efficiency on 2-D and 3-D CNNs, respectively. To improve the hardware performance, we propose a hardware-friendly quantization approach called static block floating point (BFP), which eliminates the frequent representation conversions required in traditional dynamic BFP arithmetic. Comparing with the integer linear quantization using zero-point, the static BFP quantization can decrease the logic resource consumption of the convolutional kernel design by nearly 50% on a field-programmable gate array (FPGA). Without time-consuming retraining, the proposed static BFP quantization is able to quantize the precision to 8-bit mantissa with negligible accuracy loss. As different CNNs on our reconfigurable system require different hardware and software parameters to achieve optimal hardware performance and accuracy, we also propose an automatic tool for parameter optimization. Based on our hardware design and optimization, we demonstrate that the proposed accelerator can achieve 3.8-5.6 times higher energy efficiency than graphics processing unit (GPU) implementation. Comparing with the state-of-the-art FPGA-based accelerators, our design achieves higher generality and up to 1.4-2.2 times higher resource efficiency on both 2-D and 3-D CNNs.

FPGA-based Acceleration of Davidon-Fletcher-Powell Quasi-Newton Optimization Method

Towards Lattice Quantum Chromodynamics on FPGA devices

Efficient FPGA-Based Accelerator of the L-BFGS Algorithm for IoT Applications.

An FPGA Implementation of Quadratic Programming

Acceleration of Multi-body Molecular Dynamics with Customized Parallel Dataflow

Optimizing FPGA-Based DNN Accelerator with Shared Exponential Floating-Point Format

A Ubiquitous Machine Learning Accelerator With Automatic Parallelization on FPGA

A neural network accelerated optimization method for FPGA

DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs

Field Programmable Gate Arrays for Enhancing the Speed and Energy Efficiency of Quantum Dynamics Simulations

A Low Cost High Performance Polynomial Multiplier Design for FPGA Implementation

A FPGA Placement Algorithm Based on Parallel Simulated Annealing

A Low-Power Hardware-Friendly Optimisation Algorithm With Absolute Numerical Stability and Convergence Guarantees

Design Exploration of Multi-FPGAs for Accelerating Deep Learning

FPGA-Placement via Quantum Annealing

Efficient Ab-Initio Molecular Dynamic Simulations by Offloading Fast Fourier Transformations to FPGAs

Toward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs

Hexagonal Tiling Based Multiple FPGAs Stencil Computation Acceleration and Optimization Methodology.

High-Performance Acceleration of 2-D and 3-D CNNs on FPGAs Using Static Block Floating Point

DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference

Integrating FPGA-based hardware acceleration with relational databases