Abstract:Summary Convolutional neural networks (CNNs) have shown remarkable advantages in a wide range of domains at the expense of huge parameters and computations. Modern CNNs still tend to be more complex and larger to achieve better inference accuracy. However, the complex and large structures of CNNs could slow down the inference speed. Recently, Compressing the convolutional weights to be sparse by pruning the unimportant parameters has been demonstrated as an efficient way to reduce the computations of CNNs. On the other hand, field‐programmable gate arrays (FPGAs) have been a popular hardware platform to accelerate CNN inference. In this paper, we propose an algorithm/hardware co‐optimized method for accelerating CNN inference on FPGAs. For the algorithm, we take advantage of unstructured and structured parameter sparsifying methods to achieve high sparsity and keep the regularity of convolutional weights. Correspondingly, hardware‐friendly index representations of sparse convolutional weights are proposed. For the hardware architecture, we propose row‐wise input‐stationary dataflow, which is tightly coupled with the algorithm. A row‐wise computing engine (RConv Engine) is proposed, which is based on the dataflow. Inside the RConv Engine, the scalar‐vector structure is applied to implement the basic processing elements (PEs). To flexibly calculate the feature map with various sizes, the PEs are organized in a 2D structure with two work modes. The experimental results demonstrate that our co‐optimized method implements high sparsity of convolutional weights, and the computing engine achieves high computation efficiency. Compared with other accelerators, our co‐optimized method implements a 10.9× speedup on FPS at most with the highest sparsity of convolutional weights and negligible accuracy loss.

Optimizing Half Precision Winograd Convolution on ARM Many-Core Processors

EWS: an Energy-Efficient CNN Accelerator with Enhanced Weight Stationary Dataflow

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

NUMA-aware FFT-based Convolution on ARMv8 Many-core CPUs

High Performance Depthwise and Pointwise Convolutions on Mobile Devices

ABM-SpConv-SIMD: Accelerating Convolutional Neural Network Inference for Industrial IoT Applications on Edge Devices

WinoCNN: Kernel Sharing Winograd Systolic Array for Efficient Convolutional Neural Network Acceleration on FPGAs

A High-efficiency FPGA-based Accelerator for Convolutional Neural Networks using Winograd Algorithm

Automatic Generation of High-Performance Convolution Kernels on ARM CPUs for Deep Learning

An Efficient Streaming Accelerator for Low Bit-Width Convolutional Neural Networks

A High Utilization FPGA-Based Accelerator for Variable-Scale Convolutional Neural Network

Optimization of Convolution Neural Network Algorithm Based on FPGA

BISWSRBS: A Winograd-based CNN Accelerator with a Fine-grained Regular Sparsity Pattern and Mixed Precision Quantization

Efficient Hardware Architectures for Deep Convolutional Neural Network

Energy-Efficient Architecture for FPGA-based Deep Convolutional Neural Networks with Binary Weights

An algorithm/hardware co‐optimized method to accelerate CNNs with compressed convolutional weights on FPGA

An Efficient Accelerator for Multiple Convolutions From the Sparsity Perspective

A High-Efficient and Configurable Hardware Accelerator for Convolutional Neural Network

Evaluating Fast Algorithms for Convolutional Neural Networks on FPGAs

Towards Design Space Exploration and Optimization of Fast Algorithms for Convolutional Neural Networks (CNNs) on FPGAs