Abstract:Convolution neural networks (CNNs) as one of today’s main flavor of deep learning techniques dominate in various image recognition tasks. As the model size of modern CNNs continues to grow, neural network compression techniques have been proposed to prune the redundant neurons and synapses. However, prior techniques disconnect the software neural networks compression and hardware acceleration, which fail to balance multiple design parameters, including sparsity, performance, hardware area cost, and efficiency. More concretely, prior unstructured pruning techniques achieve high sparsity at the expense of extra performance overhead, while prior structured pruning techniques relying on strict sparse patterns lead to low sparsity and extra hardware cost. In this article, we propose OMNI, a framework for accelerating sparse CNNs on hardware accelerators. The innovation of OMNI stems from that it uses hardware amenable on-chip memory partition patterns to seamlessly engage the software CNN model compression and hardware CNN acceleration. To accelerate the compute-intensive convolution kernel, a promising hardware optimization approach is memory partition, which divides the original weight kernels into several groups so that the different hardware processing elements can simultaneously access the weight. We exploit the memory partition patterns including block, cyclic, or hybrid as a means of CNN compression patterns. Our software CNN model compression balances the sparsity across different groups and our hardware accelerator employs hardware parallelization coordinately with the sparse patterns, leading to a desirable compromise between sparsity and performance. We further develop performance models to help the designers to quickly identify the pattern factors subject to an area constraint. Last, we evaluate our design on application specific integrated circuit (ASIC) and field-programmable gate array (FPGA) platform. Experiments demonstrate that OMNI achieves $3.4\times $ – $6.2\times $ speedup for the modern CNNs, over a comparably ideal dense CNN accelerator. OMNI shows $114.7\times $ energy efficiency improvement compared with GPU platform. OMNI is also evaluated on Xilinx ZC706 and ZCU102 FPGA platforms, achieving 41.5 GOP/s and 125.3 GOP/s, respectively.

Sparse Systolic Tensor Array for Efficient CNN Hardware Acceleration

An Efficient CNN Training Accelerator Leveraging Transposable Block Sparsity

An Efficient and Flexible Accelerator Design for Sparse Convolutional Neural Networks

An Efficient Accelerator for Multiple Convolutions From the Sparsity Perspective

Sense: Model Hardware Co-design for Accelerating Sparse CNN on Systolic Array

Accelerator for Sparse Convolutional Neural Networks Based on Shift Units

Sense: Model-Hardware Codesign for Accelerating Sparse CNNs on Systolic Arrays

A Systolic Array-Based Scheduling Strategy for Sparse CNN Accelerators

A Balanced Sparse Matrix Convolution Accelerator for Efficient CNN Training

SpWMM: A High-Performance Sparse-Winograd Matrix-Matrix Multiplication Accelerator for CNNs.

A Reconfigurable Accelerator for Sparse Convolutional Neural Networks.

AdaS: A Fast and Energy-Efficient CNN Accelerator Exploiting Bit-Sparsity

A Fine-Grained Sparse Accelerator for Multi-Precision DNN.

OMNI: A Framework for Integrating Hardware and Software Optimizations for Sparse CNNs

Eyelet: A Cross-Mesh NoC-Based Fine-Grained Sparse CNN Accelerator for Spatio-Temporal Parallel Computing Optimization

A Computing Efficient Hardware Architecture for Sparse Deep Neural Network Computing

A Low-Power Sparse Convolutional Neural Network Accelerator with Pre-Encoding Radix-4 Booth Multiplier

SPRING: A Sparsity-Aware Reduced-Precision Monolithic 3D CNN Accelerator Architecture for Training and Inference

Relative Indexed Compressed Sparse Filter Encoding Format for Hardware-Oriented Acceleration of Deep Convolutional Neural Networks

A High-Performance Systolic Array Accelerator Dedicated for CNN.

Reconfigurable Spatial-Parallel Stochastic Computing for Accelerating Sparse Convolutional Neural Networks