Abstract:Fine-grained sparse convolutional neural networks (CNNs) achieve a better trade-off between model accuracy and size than coarse-grained sparse CNNs. Due to irregular data structures and unbalanced computation loads, fine-grained sparse CNNs struggle to fully leverage the performance advantages of computation and storage on general-purpose edge hardware. However, existing custom sparse accelerators are designed from the perspective of emulating a balanced load by software or computational strategies, neglecting the exploration of the computing architecture’s adaptability and parallelism for fine-grained sparse models. To address these challenges, a cross-mesh NoC-based accelerator architecture is proposed. This architecture aligns with the irregular characteristics of fine-grained sparse CNN weights and enhances the spatio-temporal parallelism of fine-grained sparse CNNs. First, a sparse multiplier unit (SMU) array and an adder array are designed to enable parallel execution of convolution multiplication and accumulation operations. Then, element-wise unroll-based nonzero weight multiplication is mapped to the SMU array to provide more flexible spatial parallelism. A horizontal and vertical cross-mesh NoC is proposed for flexible dataflow scheduling between the SMU and adder arrays to further improve temporal parallelism. This architecture allows the multiplication and accumulation operations in convolution to be decoupled and pipelined with negligible latency. Finally, the proposed accelerator architecture is implemented on the ZU9EG platform. The experimental results show that the proposed accelerator achieves frame rates of 509.9, 249.3, 100.7, 48.4, and 168.9 frames per second (FPS) for AlexNet, VGG-16, ResNet-18, MobileNet-v2, and EfficientNet, respectively. Compared with related works, this accelerator achieves inference speed and energy efficiency improvements of 1.1 $\times$ $\sim$ 36.1 $\times$ and 2.4 $\times$ $\sim$ 13.4 $\times$ , respectively.

Reconfigurable Spatial-Parallel Stochastic Computing for Accelerating Sparse Convolutional Neural Networks

Optimizing Stochastic Computing for Low Latency Inference of Convolutional Neural Networks

Parallel Convolutional Neural Network (CNN) Accelerators Based on Stochastic Computing

Hybrid Stochastic-Binary Computing for Low-Latency and High-Precision Inference of CNNs

Towards Acceleration of Deep Convolutional Neural Networks Using Stochastic Computing

An Efficient Accelerator for Multiple Convolutions From the Sparsity Perspective

A Reconfigurable Accelerator for Sparse Convolutional Neural Networks.

An Efficient and Flexible Accelerator Design for Sparse Convolutional Neural Networks

SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks

Stochastic-Binary Hybrid Spatial Coding Multiplier for Convolutional Neural Network Accelerator

An Efficient Stochastic Convolution Accelerator Based on Pseudo-Sobol Sequences.

Efficient Non-Linear Adder for Stochastic Computing with Approximate Spatial-Temporal Sorting Network

Accelerator for Sparse Convolutional Neural Networks Based on Shift Units

A Convolutional Spiking Neural Network Accelerator with the Sparsity-Aware Memory and Compressed Weights

SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing

WRA-SS: A High-Performance Accelerator Integrating Winograd with Structured Sparsity for Convolutional Neural Networks

Not Your Father’s Stochastic Computing (SC)! Efficient Yet Accurate End-to-End SC Accelerator Design

Stochastic Computing Hardware Design and Optimization for Convolutional Neutral Networks

Eyelet: A Cross-Mesh NoC-Based Fine-Grained Sparse CNN Accelerator for Spatio-Temporal Parallel Computing Optimization

VSCNN: Convolution Neural Network Accelerator With Vector Sparsity

Search-Free Inference Acceleration for Sparse Convolutional Neural Networks