Abstract:In recent years, convolutional neural network (CNN) based methods have achieved great success in a large number of applications and have been among the most powerful and widely used techniques in computer vision. However, CNN-based methods are com-putational-intensive and resource-consuming, and thus are hard to be integrated into embedded systems such as smart phones, smart glasses, and robots. FPGA is one of the most promising platforms for accelerating CNN, but the limited bandwidth and on-chip memory size limit the performance of FPGA accelerator for CNN. In this paper, we go deeper with the embedded FPGA platform on accelerating CNNs and propose a CNN accelerator design on embedded FPGA for Image-Net large-scale image classification. We first present an in-depth analysis of state-of-the-art CNN models and show that Convolutional layers are computational-centric and Fully-Connected layers are memory-centric. Then the dynamic-precision data quantization method and a convolver design that is efficient for all layer types in CNN are proposed to improve the bandwidth and resource utilization. Results show that only 0.4% accuracy loss is introduced by our data quantization flow for the very deep VGG16 model when 8/4-bit quantization is used. A data arrangement method is proposed to further ensure a high utilization of the external memory bandwidth. Finally, a state-of-the-art CNN, VGG16-SVD, is implemented on an embedded FPGA platform as a case study. VGG16-SVD is the largest and most accurate network that has been implemented on FPGA end-to-end so far. The system on Xilinx Zynq ZC706 board achieves a frame rate at 4.45 fps with the top-5 accuracy of 86.66% using 16-bit quantization. The average performance of convolutional layers and the full CNN is 187.8 GOP/s and 137.0 GOP/s under 150MHz working frequency, which outperform previous approaches significantly.

VarGNet: Variable Group Convolutional Neural Network for Efficient Embedded Computing

IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

Interleaved Group Convolutions for Deep Neural Networks

SPGNet: Serial and Parallel Group Network

ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions

CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

SRGC-Nets: Sparse Repeated Group Convolutional Neural Networks

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Exploring Hardware Friendly Bottleneck Architecture In Cnn For Embedded Computing Systems

Variational Gridded Graph Convolution Network for Node Classification

Effnet: An Efficient Structure for Convolutional Neural Networks

GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological Interpretability

Going Deeper with Embedded FPGA Platform for Convolutional Neural Network

Efficient Equivariant Network

Low power Convolutional Neural Networks on a chip

CSFNet: a compact and efficient convolution-transformer hybrid vision model

A Reconfigurable Spatial Architecture for Energy-Efficient Inception Neural Networks

OGCNet: Overlapped group convolution for deep convolutional neural networks

Moving Convolutional Neural Networks to Embedded Systems: The AlexNet and VGG-16 Case

Efficient Hardware Architectures for Deep Convolutional Neural Network