Abstract:With the rapid development and continuous evolution of convolutional neural networks (CNNs), FPGAs have become one of the most attractive candidates for deploying CNNs due to their re-programmability, low power consumption, and fast time-to-market characteristics. However, as the network structure of models deepens, previous FPGA solutions based on the traditional convolution are still limited by computational power, making it challenging to meet feedforward performance requirements. In this article, we introduce the state-of-the-art octave convolution (OctConv) into the CNN accelerator design for the first time to improve the hardware acceleration efficiency and design a dedicated OctPU for mapping OctConv to FPGAs efficiently, which employs a parallel dataflow pattern to exploit the parallelism of OctConv sufficiently. Based on this, we present a novel and scalable architecture that dynamically combines the inter-layer pipelined structure and multi-layer reuse structure, achieving a compromise between specificity and scalability with limited resources. Meanwhile, to obtain the optimized solution from the complex design space search, we build a multidimensional performance and resource analysis model and a two-stage search algorithm based on greedy and heuristic algorithms. We evaluate our proposal by implementing VGG16 and ResNet50 on the Xilinx VU9P FPGA. Experimental results show that our prototype accelerators can achieve an average of 3321 GOP/s for the convolutional layers for VGG16 and 2873 GOP/s for the overall ResNet50 using OctConv. Compared to previous works based on the traditional convolution, our prototypes own a 1.72× to 2.33× speedup in throughput and a 2.01× to 5.18× improvement in computational density. Our design also presents an excellent compromise performance and generalization compared to previous hardware and software co-optimization works.

High Performance Robust Audio Event Recognition System Based on FPGA Platform.

A High Performance FPGA-Based Accelerator Design for End-to-End Speaker Recognition System

Face Recognition with Hybrid Efficient Convolution Algorithms on FPGAs

Design and Implementation of Intelligent Speech Recognition System Based on FPGA

A High Performance FPGA-based Accelerator for Large-Scale Convolutional Neural Networks

Going Deeper with Embedded FPGA Platform for Convolutional Neural Network

A High-Performance Accelerator for Large-Scale Convolutional Neural Networks

A Grain-Adaptive Computing Structure for FPGA CNN Acceleration.

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.

A High-performance Hardware Accelerator Using a Fusion Approach of Convolution and Pooling

High Throughput CNN Accelerator Design Based on FPGA

Implementation of Convolutional Neural Network with Co-design of High-Level Synthesis and Verilog HDL

CNN-based Feature-point Extraction for Real-time Visual SLAM on Embedded FPGA

OctCNN: A High Throughput FPGA Accelerator for CNNs using Octave Convolution Algorithm

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

High Power-Efficient and Performance-Density FPGA Accelerator for CNN-Based Object Detection

WPU: A FPGA-based Scalable, Efficient and Software/Hardware Co-design Deep Neural Network Inference Acceleration Processor

Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application

An FPGA-Based CNN Accelerator Integrating Depthwise Separable Convolution

A Scalable FPGA Accelerator for Convolutional Neural Networks.