Abstract:Fine-grained sparse convolutional neural networks (CNNs) achieve a better trade-off between model accuracy and size than coarse-grained sparse CNNs. Due to irregular data structures and unbalanced computation loads, fine-grained sparse CNNs struggle to fully leverage the performance advantages of computation and storage on general-purpose edge hardware. However, existing custom sparse accelerators are designed from the perspective of emulating a balanced load by software or computational strategies, neglecting the exploration of the computing architecture’s adaptability and parallelism for fine-grained sparse models. To address these challenges, a cross-mesh NoC-based accelerator architecture is proposed. This architecture aligns with the irregular characteristics of fine-grained sparse CNN weights and enhances the spatio-temporal parallelism of fine-grained sparse CNNs. First, a sparse multiplier unit (SMU) array and an adder array are designed to enable parallel execution of convolution multiplication and accumulation operations. Then, element-wise unroll-based nonzero weight multiplication is mapped to the SMU array to provide more flexible spatial parallelism. A horizontal and vertical cross-mesh NoC is proposed for flexible dataflow scheduling between the SMU and adder arrays to further improve temporal parallelism. This architecture allows the multiplication and accumulation operations in convolution to be decoupled and pipelined with negligible latency. Finally, the proposed accelerator architecture is implemented on the ZU9EG platform. The experimental results show that the proposed accelerator achieves frame rates of 509.9, 249.3, 100.7, 48.4, and 168.9 frames per second (FPS) for AlexNet, VGG-16, ResNet-18, MobileNet-v2, and EfficientNet, respectively. Compared with related works, this accelerator achieves inference speed and energy efficiency improvements of 1.1 $\times$ $\sim$ 36.1 $\times$ and 2.4 $\times$ $\sim$ 13.4 $\times$ , respectively.

A 65-Nm Energy-Efficient Interframe Data Reuse Neural Network Accelerator for Video Applications

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

An Energy-Efficient Differential Frame Convolutional Accelerator with On-Chip Fusion Storage Architecture and Pixel-Level Pipeline Data Flow

14.2 A 65nm 24.7 µJ/Frame 12.3 mW Activation-Similarity-Aware Convolutional Neural Network Video Processor Using Hybrid Precision, Inter-Frame Data Reuse and Mixed-Bit-Width …

14.2 A 65nm 24.7µj/frame 12.3mw Activation-Similarity-Aware Convolutional Neural Network Video Processor Using Hybrid Precision, Inter-Frame Data Reuse and Mixed-Bit-Width Difference-Frame Data Codec

An Efficient CNN Accelerator Using Inter-Frame Data Reuse of Videos on FPGAs

A Computationally Efficient Neural Video Compression Accelerator Based on a Sparse CNN-Transformer Hybrid Network

An Efficient Streaming Accelerator for Low Bit-Width Convolutional Neural Networks

Eyelet: A Cross-Mesh NoC-Based Fine-Grained Sparse CNN Accelerator for Spatio-Temporal Parallel Computing Optimization

An Efficient and Flexible Accelerator Design for Sparse Convolutional Neural Networks

A Precision-Scalable Energy-Efficient Convolutional Neural Network Accelerator.

An Efficient Accelerator for Multiple Convolutions From the Sparsity Perspective

A Reconfigurable Accelerator for Sparse Convolutional Neural Networks.

An Asynchronous Energy-Efficient CNN Accelerator with Reconfigurable Architecture.

An Efficient CNN Accelerator Achieving High PE Utilization Using a Dense-/Sparse-Aware Redundancy Reduction Method and Data–Index Decoupling Workflow

An Efficient FPGA-based Depthwise Separable Convolutional Neural Network Accelerator with Hardware Pruning.

A 681 GOPS/W~3.59 TOPS/W CNN Accelerator Based on Novel Data Flow Scheduling Scheme

Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks

Design of a Convolutional Neural Network Accelerator Based on On-Chip Data Reordering

A High-performance Inference Accelerator Exploiting Patterned Sparsity in CNNs

A Balanced Sparse Matrix Convolution Accelerator for Efficient CNN Training