Abstract:In recent years, convolutional neural networks (CNNs) have demonstrated their ability to solve problems in many fields and with accuracy that was not possible before. However, this comes with extensive computational requirements, which made general CPUs unable to deliver the desired real-time performance. At the same time, FPGAs have seen a surge in interest for accelerating CNN inference. This is due to their ability to create custom designs with different levels of parallelism. Furthermore, FPGAs provide better performance per watt compared to GPUs. The current trend in FPGA-based CNN accelerators is to implement multiple convolutional layer processors (CLPs), each of which is tailored for a subset of layers. However, the growing complexity of CNN architectures makes optimizing the resources available on the target FPGA device to deliver optimal performance more challenging. In this paper, we present a CNN accelerator and an accompanying automated design methodology that employs metaheuristics for partitioning available FPGA resources to design a Multi-CLP accelerator. Specifically, the proposed design tool adopts simulated annealing (SA) and tabu search (TS) algorithms to find the number of CLPs required and their respective configurations to achieve optimal performance on a given target FPGA device. Here, the focus is on the key specifications and hardware resources, including digital signal processors, block RAMs, and off-chip memory bandwidth. Experimental results and comparisons using four well-known benchmark CNNs are presented demonstrating that the proposed acceleration framework is both encouraging and promising. The SA-/TS-based Multi-CLP achieves 1.31x - 2.37x higher throughput than the state-of-the-art Single-/Multi-CLP approaches in accelerating AlexNet, SqueezeNet 1.1, VGGNet, and GoogLeNet architectures on the Xilinx VC707 and VC709 FPGA boards.

Hybrid CNN-SVM Inference Accelerator on FPGA Using HLS

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Design of FPGA-Based Accelerator for Convolutional Neural Network under Heterogeneous Computing Framework with OpenCL

An Efficient Sparse CNNs Accelerator on FPGA

A FPGA-based Hardware Accelerator for Multiple Convolutional Neural Networks

Optimizing CNN-based Hyperspectral Image Classification on FPGAs

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs

A high throughput acceleration for hybrid neural networks with efficient resource management on FPGA

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

Optimization of FPGA-based CNN Accelerators Using Metaheuristics

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

A flexible FPGA accelerator for convolutional neural networks

An FPGA-Based Accelerator Enabling Efficient Support for CNNs with Arbitrary Kernel Sizes

A Flexible and Efficient FPGA Accelerator for Various Large-Scale and Lightweight CNNs

FPGA-based Accelerator for Convolutional Neural Network

An FPGA-based HEVC post-processing CNN hardware accelerator

A High-Performance Accelerator for Large-Scale Convolutional Neural Networks

Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs.

A Power-Efficient and High Performance FPGA Accelerator for Convolutional Neural Networks: Work-in-progress.

Efficient Inference of Large-Scale and Lightweight Convolutional Neural Networks on FPGA