Abstract:Convolutional neural networks (CNNs) are to be effective in many application domains, especially in the computer vision area. In order to achieve lower latency CNN processing, and reduce power consumption, developers are experimenting with using FPGAs to accelerate CNN processing in several applications. Current FPGA CNN accelerators usually use the same acceleration approaches as GPUs, where operations from different network layers are mapped to the same hardware units working in a multiplexed manner. This will result in high flexibility in implementing different types of CNNs; however, this will degrade the latency that accelerators can achieve. Alternatively, we can reduce the latency of the accelerator by pipelining the processing of consecutive layers, at the expense of more FPGA resources. The continued increase in hardware resources available in FPGAs makes such implementations feasible for latency-critical application domains. In this paper, we present FPQNet, a fully pipelined and quantized CNN FPGA implementation that is channel-parallel, layer-pipelined, and network-parallel, to decrease latency and increase throughput, combined with quantization methods to optimize hardware utilization. In addition, we optimize this hardware architecture for the HDMI timing standard to avoid extra hardware utilization. This makes it possible for the accelerator to handle video datasets. We present prototypes of the FPQNet CNN network implementations on an Alpha Data 9H7 FPGA, connected with an OpenCAPI interface, to demonstrate architecture capabilities. Results show that with a 250 MHz clock frequency, an optimized LeNet-5 design is able to achieve latencies as low as 9.32 µs with an accuracy of 98.8% on the MNIST dataset, making it feasible for utilization in high frame rate video processing applications. With 10 hardware kernels working concurrently, the throughput is as high as 1108 GOPs. The methods in this paper are suitable for many other CNNs. Our analysis shows that the latency of AlexNet, ZFNet, OverFeat-Fast, and OverFeat-Accurate can be as low as 69.27, 66.95, 182.98, and 132.6 µs, using the architecture introduced in this paper, respectively.

Fast Neural Network Training on FPGA Using Quasi-Newton Optimization Method

A Multi-Level-Optimization Framework for FPGA-Based Cellular Neural Network Implementation.

Speed and Resource Optimization of BFGS Quasi-Newton Implementation on FPGA Using Inexact Line Search Method for Neural Network Training

FPGA based cellular neural network optimization: from design space to system.

FPGA-based Acceleration of Neural Network Training

A neural network accelerated optimization method for FPGA

FPGA-based Acceleration of Davidon-Fletcher-Powell Quasi-Newton Optimization Method

A Learning Framework for n-bit Quantized Neural Networks toward FPGAs

FPGA Implementation of Quantized Convolutional Neural Networks.

A Power Efficient Neural Network Implementation on Heterogeneous FPGA and GPU Devices

FPQNet: Fully Pipelined and Quantized CNN for Ultra-Low Latency Image Classification on FPGAs Using OpenCAPI

Explore Efficient LUT-based Architecture for Quantized Convolutional Neural Networks on FPGA

Edge FPGA-based Onsite Neural Network Training.

Neural Networks Hardware Implementation Based on FPGA

Genetic Algorithm Augmented Inception-Net based Image Classifier Accelerated on FPGA

Acceleration of Deep Neural Network Training Using Field Programmable Gate Arrays

Resource Reduction of BFGS Quasi-Newton Implementation on FPGA Using Fixed-Point Matrix Updating

FPGA-based Acceleration of Deep Neural Networks Using High Level Method.

FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks

A Reconfigurable DNN Training Accelerator on FPGA

Tiny Neural Network Search and Implementation for Embedded FPGA: a Software-Hardware Co-Design Approach