Abstract:In recent years, convolutional neural network (CNN) based methods have achieved great success in a large number of applications and have been among the most powerful and widely used techniques in computer vision. However, CNN-based methods are com-putational-intensive and resource-consuming, and thus are hard to be integrated into embedded systems such as smart phones, smart glasses, and robots. FPGA is one of the most promising platforms for accelerating CNN, but the limited bandwidth and on-chip memory size limit the performance of FPGA accelerator for CNN. In this paper, we go deeper with the embedded FPGA platform on accelerating CNNs and propose a CNN accelerator design on embedded FPGA for Image-Net large-scale image classification. We first present an in-depth analysis of state-of-the-art CNN models and show that Convolutional layers are computational-centric and Fully-Connected layers are memory-centric. Then the dynamic-precision data quantization method and a convolver design that is efficient for all layer types in CNN are proposed to improve the bandwidth and resource utilization. Results show that only 0.4% accuracy loss is introduced by our data quantization flow for the very deep VGG16 model when 8/4-bit quantization is used. A data arrangement method is proposed to further ensure a high utilization of the external memory bandwidth. Finally, a state-of-the-art CNN, VGG16-SVD, is implemented on an embedded FPGA platform as a case study. VGG16-SVD is the largest and most accurate network that has been implemented on FPGA end-to-end so far. The system on Xilinx Zynq ZC706 board achieves a frame rate at 4.45 fps with the top-5 accuracy of 86.66% using 16-bit quantization. The average performance of convolutional layers and the full CNN is 187.8 GOP/s and 137.0 GOP/s under 150MHz working frequency, which outperform previous approaches significantly.

INCAME: INterruptible CNN Accelerator for Multi-robot Exploration.

INCA: INterruptible CNN Accelerator for Multi-tasking in Embedded Robots

A FPGA-based Accelerator of Convolutional Neural Network for Face Feature Extraction

CaFPGA: an Automatic Generation Model for CNN Accelerator.

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Scalable FPGA-Based Convolutional Neural Network Accelerator for Embedded Systems

MALOC: A Fully Pipelined FPGA Accelerator for Convolutional Neural Networks with All Layers Mapped on Chip

Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application

Going Deeper with Embedded FPGA Platform for Convolutional Neural Network

A Memory-Optimized and Energy-Efficient CNN Acceleration Architecture Based on FPGA.

A flexible FPGA accelerator for convolutional neural networks

Toward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs

FPGA-based Accelerator for Convolutional Neural Network

A FPGA-based end-to-end acceleration framework for fast deployment of Convolutional Neural Networks

Communication-Aware and Resource-Efficient NoC-Based Architecture for CNN Acceleration

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

A High-performance Hardware Accelerator Using a Fusion Approach of Convolution and Pooling

An OpenCL-Based FPGA Accelerator for Faster R-CNN

A Power-Efficient and High Performance FPGA Accelerator for Convolutional Neural Networks: Work-in-progress.

Angel-Eye: A Complete Design Flow for Mapping CNN Onto Customized Hardware.