Abstract:In recent years, convolutional neural networks (CNNs) have demonstrated their ability to solve problems in many fields and with accuracy that was not possible before. However, this comes with extensive computational requirements, which made general CPUs unable to deliver the desired real-time performance. At the same time, FPGAs have seen a surge in interest for accelerating CNN inference. This is due to their ability to create custom designs with different levels of parallelism. Furthermore, FPGAs provide better performance per watt compared to GPUs. The current trend in FPGA-based CNN accelerators is to implement multiple convolutional layer processors (CLPs), each of which is tailored for a subset of layers. However, the growing complexity of CNN architectures makes optimizing the resources available on the target FPGA device to deliver optimal performance more challenging. In this paper, we present a CNN accelerator and an accompanying automated design methodology that employs metaheuristics for partitioning available FPGA resources to design a Multi-CLP accelerator. Specifically, the proposed design tool adopts simulated annealing (SA) and tabu search (TS) algorithms to find the number of CLPs required and their respective configurations to achieve optimal performance on a given target FPGA device. Here, the focus is on the key specifications and hardware resources, including digital signal processors, block RAMs, and off-chip memory bandwidth. Experimental results and comparisons using four well-known benchmark CNNs are presented demonstrating that the proposed acceleration framework is both encouraging and promising. The SA-/TS-based Multi-CLP achieves 1.31x - 2.37x higher throughput than the state-of-the-art Single-/Multi-CLP approaches in accelerating AlexNet, SqueezeNet 1.1, VGGNet, and GoogLeNet architectures on the Xilinx VC707 and VC709 FPGA boards.

A high throughput acceleration for hybrid neural networks with efficient resource management on FPGA

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

HybridDNN: A Framework for High-Performance Hybrid DNN Accelerator Design and Implementation.

A neural network accelerated optimization method for FPGA

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

A Hybrid Heterogeneous Neural Network Accelerator Based on Systolic Array

A High-Performance Accelerator for Large-Scale Convolutional Neural Networks

A Ubiquitous Machine Learning Accelerator With Automatic Parallelization on FPGA

A Convolutional Neural Network Accelerator Based on FPGA

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

Optimization of FPGA-based CNN Accelerators Using Metaheuristics

Design of FPGA-Based Accelerator for Convolutional Neural Network under Heterogeneous Computing Framework with OpenCL

FPGA-Based High-Throughput CNN Hardware Accelerator With High Computing Resource Utilization Ratio

MALOC: A Fully Pipelined FPGA Accelerator for Convolutional Neural Networks with All Layers Mapped on Chip

Design Flow of Accelerating Hybrid Extremely Low Bit-Width Neural Network in Embedded FPGA

Deep neural network accelerator based on FPGA

Multi-clusters: an Efficient Design Paradigm of NN Accelerator Architecture Based on FPGA

A Power-Efficient and High Performance FPGA Accelerator for Convolutional Neural Networks: Work-in-progress.

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

An All-Digital Compute-In-Memory FPGA Architecture for Deep Learning Acceleration

Design and Optimization of Residual Neural Network Accelerators for Low-Power FPGAs Using High-Level Synthesis