Abstract:Due to the huge success and rapid development of convolutional neural networks (CNNs), there is a growing demand for hardware accelerators that accommodate a variety of CNNs to improve their inference latency and energy efficiency, in order to enable their deployment in real-time applications. Among popular platforms, field-programmable gate arrays (FPGAs) have been widely adopted for CNN acceleration because of their capability to provide superior energy efficiency and low-latency processing, while supporting high reconfigurability, making them favorable for accelerating rapidly evolving CNN algorithms. This article introduces a highly customized streaming hardware architecture that focuses on improving the compute efficiency for streaming applications by providing full-stack acceleration of CNNs on FPGAs. The proposed accelerator maps most computational functions, that is, convolutional and deconvolutional layers into a singular unified module, and implements the residual and concatenative connections between the functions with high efficiency, to support the inference of mainstream CNNs with different topologies. This architecture is further optimized through exploiting different levels of parallelism, layer fusion, and fully leveraging digital signal processing blocks (DSPs). The proposed accelerator has been implemented on Intel's Arria 10 GX1150 hardware and evaluated with a wide range of benchmark models. The results demonstrate a high performance of over 1.3 TOP/s of throughput, up to 97% of compute [multiply-accumulate (MAC)] efficiency, which outperforms the state-of-the-art FPGA accelerators.

A High-Performance Systolic Array Accelerator Dedicated for CNN.

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

High-performance Convolutional Neural Network Accelerator Based on Systolic Arrays and Quantization

Dual-Line-Systolic Array for High Performance CNN Accelerator

Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs.

A High-Efficient and Configurable Hardware Accelerator for Convolutional Neural Network

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

Design of FPGA Accelerator Architecture for Convolutional Neural Network

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

A High Efficient Architecture for Convolution Neural Network Accelerator

FPGA-based Accelerator for Convolutional Neural Network

A High-performance Hardware Accelerator Using a Fusion Approach of Convolution and Pooling

A High-Performance Accelerator for Large-Scale Convolutional Neural Networks

Heterogeneous Systolic Array Architecture for Compact CNNs Hardware Accelerators

A Flexible and Efficient FPGA Accelerator for Various Large-Scale and Lightweight CNNs

LACS: A High-Computational-Efficiency Accelerator for CNNs

High-Speed CNN Accelerator SoC Design Based on a Flexible Diagonal Cyclic Array

A Block-Floating-Point Arithmetic Based FPGA Accelerator for Convolutional Neural Networks

Toward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs

An FPGA-Based Accelerator Enabling Efficient Support for CNNs with Arbitrary Kernel Sizes

Systolic Array Based Accelerator and Algorithm Mapping for Deep Learning Algorithms.