Abstract:Convolutional neural networks (CNNs) are rapidly evolving and being applied to a broad range of applications. Given a specific application, an increasing challenge is to search the appropriate CNN algorithm and efficiently map it to the target hardware. The FPGA-based accelerator has the advantage of reconfigurability and flexibility, and has achieved highperformance and low-power. Without a general compiler to automate the implementation, however, significant efforts and expertise are still required to customize the design for each CNN model. In this work, we present an RTL-level CNN compiler that automatically generates customized FPGA hardware for the inference tasks of various CNNs, in order to enable high-level fast prototyping of CNNs from software to FPGA and still keep the benefits of low-level hardware optimization. First, a generalpurpose library of RTL modules is developed to model different operations at each layer. The implementation of each module is optimized at the RTL level. Given a CNN algorithm, its structure is abstracted to a directed acyclic graph (DAG) and then complied with RTL modules in the library. The integration and dataflow of physical modules are predefined in the top-level system template and reconfigured during compilation. The runtime control of layer-by-layer sequential computation is managed by the proposed execution schedule so that even highly irregular and complex network topology, e.g. ResNet, can be compiled. The proposed methodology is demonstrated with end-to-end FPGA implementations of various CNN algorithms (e.g. NiN, VGG-16, ResNet-50, and ResNet-152) on two standalone Intel FPGAs, Stratix V and Arria 10. The performance and overhead of the automated compilation are evaluated. The compiled FPGA accelerators exhibit superior performance compared to state-ofthe- art automation-based works by >2x for various CNNs.

A General-Purpose Compiler Design for Instruction-Based AI Accelerator Implementation.

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

A Compiler Design for a Programmable CNN Accelerator

Automatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

DNNVM - End-to-End Compiler Leveraging Operation Fusion on FPGA-based CNN Accelerators.

Leveraging Bit-Serial Architectures for Hardware-Oriented Deep Learning Accelerators with Column-Buffering Dataflow

Restoring the Broken Covenant Between Compilers and Deep Learning Accelerators

Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs

FusionAccel: A General Re-configurable Deep Learning Inference Accelerator on FPGA for Convolutional Neural Networks

Automatic Compiler Based FPGA Accelerator for CNN Training

PowerFusion: A Tensor Compiler with Explicit Data Movement Description and Instruction-level Graph IR

CFEACT: A CGRA-based Framework Enabling Agile CNN and Transformer Accelerator Design

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

A flexible FPGA accelerator for convolutional neural networks

DNNVM : End-to-End Compiler Leveraging Heterogeneous Optimizations on FPGA-based CNN Accelerators

A Flexible and Energy-Efficient Convolutional Neural Network Acceleration with Dedicated ISA and Accelerator

Research on Convolutional Neural Network Inference Acceleration and Performance Optimization for Edge Intelligence

Hardware-Efficient Template-Based Deep CNNs Accelerator Design

An Automatic Rtl Compiler For High-Throughput Fpga Implementation Of Diverse Deep Convolutional Neural Networks