Abstract:Residual neural networks are widely used in computer vision tasks. They enable the construction of deeper and more accurate models by mitigating the vanishing gradient problem. Their main innovation is the residual block which allows the output of one layer to bypass one or more intermediate layers and be added to the output of a later layer. Their complex structure and the buffering required by the residual block make them difficult to implement on resource-constrained platforms. We present a novel design flow for implementing deep learning models for field programmable gate arrays optimized for ResNets, using a strategy to reduce their buffering overhead to obtain a resource-efficient implementation of the residual layer. Our high-level synthesis (HLS)-based flow encompasses a thorough set of design principles and optimization strategies, exploiting in novel ways standard techniques such as temporal reuse and loop merging to efficiently map ResNet models, and potentially other skip connection-based NN architectures, into FPGA. The models are quantized to 8-bit integers for both weights and activations, 16-bit for biases, and 32-bit for accumulations. The experimental results are obtained on the CIFAR-10 dataset using ResNet8 and ResNet20 implemented with Xilinx FPGAs using HLS on the Ultra96-V2 and Kria KV260 boards. Compared to the state-of-the-art on the Kria KV260 board, our ResNet20 implementation achieves 2.88X speedup with 0.5% higher accuracy of 91.3%, while ResNet8 accuracy improves by 2.8% to 88.7%. The throughputs of ResNet8 and ResNet20 are 12971 FPS and 3254 FPS on the Ultra96 board, and 30153 FPS and 7601 FPS on the Kria KV26, respectively. They Pareto-dominate state-of-the-art solutions concerning accuracy, throughput, and energy.

End-to-end Scalable FPGA Accelerator for Deep Residual Networks.

A Deep Residual Networks Accelerator on FPGA

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

FPGA accelerates deep residual learning for image recognition

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

Design and Optimization of Residual Neural Network Accelerators for Low-Power FPGAs Using High-Level Synthesis

Deep neural network accelerator based on FPGA

A Scalable FPGA Accelerator for Convolutional Neural Networks.

FPGA Hardware Acceleration Design for Deep Learning

A FPGA-based end-to-end acceleration framework for fast deployment of Convolutional Neural Networks

Scalable FPGA-Based Convolutional Neural Network Accelerator for Embedded Systems

An OpenCL-Based FPGA Accelerator for Faster R-CNN

High Throughput CNN Accelerator Design Based on FPGA

A High-Performance Accelerator for Real-Time Super-Resolution on Edge FPGAs

Throughput-Optimized FPGA Accelerator for Deep Convolutional Neural Networks

Optimizing the Convolution Operation to Accelerate Deep Neural Networks on FPGA

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

Efficient Inference of Large-Scale and Lightweight Convolutional Neural Networks on FPGA

An FPGA-Based Energy-Efficient Reconfigurable Convolutional Neural Network Accelerator for Object Recognition Applications

Optimizing Loop Operation and Dataflow in Fpga Acceleration of Deep Convolutional Neural Networks