Abstract:Convolutional neural networks (CNNs) are rapidly evolving and being applied to a broad range of applications. Given a specific application, an increasing challenge is to search the appropriate CNN algorithm and efficiently map it to the target hardware. The FPGA-based accelerator has the advantage of reconfigurability and flexibility, and has achieved highperformance and low-power. Without a general compiler to automate the implementation, however, significant efforts and expertise are still required to customize the design for each CNN model. In this work, we present an RTL-level CNN compiler that automatically generates customized FPGA hardware for the inference tasks of various CNNs, in order to enable high-level fast prototyping of CNNs from software to FPGA and still keep the benefits of low-level hardware optimization. First, a generalpurpose library of RTL modules is developed to model different operations at each layer. The implementation of each module is optimized at the RTL level. Given a CNN algorithm, its structure is abstracted to a directed acyclic graph (DAG) and then complied with RTL modules in the library. The integration and dataflow of physical modules are predefined in the top-level system template and reconfigured during compilation. The runtime control of layer-by-layer sequential computation is managed by the proposed execution schedule so that even highly irregular and complex network topology, e.g. ResNet, can be compiled. The proposed methodology is demonstrated with end-to-end FPGA implementations of various CNN algorithms (e.g. NiN, VGG-16, ResNet-50, and ResNet-152) on two standalone Intel FPGAs, Stratix V and Arria 10. The performance and overhead of the automated compilation are evaluated. The compiled FPGA accelerators exhibit superior performance compared to state-ofthe- art automation-based works by >2x for various CNNs.

Automatic code generation of convolutional neural networks in FPGA implementation

CaFPGA: an Automatic Generation Model for CNN Accelerator.

Automatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators

An Automatic Rtl Compiler For High-Throughput Fpga Implementation Of Diverse Deep Convolutional Neural Networks

Implementation of Convolutional Neural Network with Co-design of High-Level Synthesis and Verilog HDL

Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs.

Accelerating convolutional neural networks on FPGAs

A Self-adaptation Method of Fitting Convolutional Neural Network into FPGA: Abstract Only).

Scalable and Modularized RTL Compilation of Convolutional Neural Networks Onto FPGA

CNN2Gate: Toward Designing a General Framework for Implementation of Convolutional Neural Networks on FPGA

Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs

A Scalable FPGA Accelerator for Convolutional Neural Networks.

Energy-Efficient And High-Throughput Fpga-Based Accelerator For Convolutional Neural Networks

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

An Efficient Design Flow for Accelerating Complicated-connected CNNs on a Multi-FPGA Platform.

Throughput-Optimized FPGA Accelerator for Deep Convolutional Neural Networks

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

A High Energy Efficiency and Low Resource Consumption FPGA Accelerator for Convolutional Neural Network

An FPGA-based Processor for Training Convolutional Neural Networks

Efficient Inference of Large-Scale and Lightweight Convolutional Neural Networks on FPGA

A FPGA-based Accelerator of Convolutional Neural Network for Face Feature Extraction