Abstract:Fine-grained sparse convolutional neural networks (CNNs) achieve a better trade-off between model accuracy and size than coarse-grained sparse CNNs. Due to irregular data structures and unbalanced computation loads, fine-grained sparse CNNs struggle to fully leverage the performance advantages of computation and storage on general-purpose edge hardware. However, existing custom sparse accelerators are designed from the perspective of emulating a balanced load by software or computational strategies, neglecting the exploration of the computing architecture’s adaptability and parallelism for fine-grained sparse models. To address these challenges, a cross-mesh NoC-based accelerator architecture is proposed. This architecture aligns with the irregular characteristics of fine-grained sparse CNN weights and enhances the spatio-temporal parallelism of fine-grained sparse CNNs. First, a sparse multiplier unit (SMU) array and an adder array are designed to enable parallel execution of convolution multiplication and accumulation operations. Then, element-wise unroll-based nonzero weight multiplication is mapped to the SMU array to provide more flexible spatial parallelism. A horizontal and vertical cross-mesh NoC is proposed for flexible dataflow scheduling between the SMU and adder arrays to further improve temporal parallelism. This architecture allows the multiplication and accumulation operations in convolution to be decoupled and pipelined with negligible latency. Finally, the proposed accelerator architecture is implemented on the ZU9EG platform. The experimental results show that the proposed accelerator achieves frame rates of 509.9, 249.3, 100.7, 48.4, and 168.9 frames per second (FPS) for AlexNet, VGG-16, ResNet-18, MobileNet-v2, and EfficientNet, respectively. Compared with related works, this accelerator achieves inference speed and energy efficiency improvements of 1.1 $\times$ $\sim$ 36.1 $\times$ and 2.4 $\times$ $\sim$ 13.4 $\times$ , respectively.

S2Engine: A Novel Systolic Architecture for Sparse Convolutional Neural Networks

EWS: an Energy-Efficient CNN Accelerator with Enhanced Weight Stationary Dataflow

Sense: Model-Hardware Codesign for Accelerating Sparse CNNs on Systolic Arrays

Sense: Model Hardware Co-design for Accelerating Sparse CNN on Systolic Array

Relative Indexed Compressed Sparse Filter Encoding Format for Hardware-Oriented Acceleration of Deep Convolutional Neural Networks

COSY: an Energy-Efficient Hardware Architecture for Deep Convolutional Neural Networks Based on Systolic Array.

FSA: A Fine-Grained Systolic Accelerator for Sparse CNNs

A Computing Efficient Hardware Architecture for Sparse Deep Neural Network Computing

Stacked Filters Stationary Flow For Hardware-Oriented Acceleration Of Deep Convolutional Neural Networks

A Software/Hardware Co-design Local Irregular Sparsity Method for Accelerating CNNs on FPGA.

A Systolic SNN Inference Accelerator and Its Co-optimized Software Framework

Sparse Systolic Tensor Array for Efficient CNN Hardware Acceleration

A High-Performance Systolic Array Accelerator Dedicated for CNN.

SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training

Accelerating Convolutional Neural Network Inference Based on a Reconfigurable Sliced Systolic Array

Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization

SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks

Eyelet: A Cross-Mesh NoC-Based Fine-Grained Sparse CNN Accelerator for Spatio-Temporal Parallel Computing Optimization

A Systolic Array-Based Scheduling Strategy for Sparse CNN Accelerators

USCA: A Unified Systolic Convolution Array Architecture for Accelerating Sparse Neural Network

A GPU-based high-performance optimization method of sparse convolutional neural networks