Abstract:Fine-grained sparse convolutional neural networks (CNNs) achieve a better trade-off between model accuracy and size than coarse-grained sparse CNNs. Due to irregular data structures and unbalanced computation loads, fine-grained sparse CNNs struggle to fully leverage the performance advantages of computation and storage on general-purpose edge hardware. However, existing custom sparse accelerators are designed from the perspective of emulating a balanced load by software or computational strategies, neglecting the exploration of the computing architecture’s adaptability and parallelism for fine-grained sparse models. To address these challenges, a cross-mesh NoC-based accelerator architecture is proposed. This architecture aligns with the irregular characteristics of fine-grained sparse CNN weights and enhances the spatio-temporal parallelism of fine-grained sparse CNNs. First, a sparse multiplier unit (SMU) array and an adder array are designed to enable parallel execution of convolution multiplication and accumulation operations. Then, element-wise unroll-based nonzero weight multiplication is mapped to the SMU array to provide more flexible spatial parallelism. A horizontal and vertical cross-mesh NoC is proposed for flexible dataflow scheduling between the SMU and adder arrays to further improve temporal parallelism. This architecture allows the multiplication and accumulation operations in convolution to be decoupled and pipelined with negligible latency. Finally, the proposed accelerator architecture is implemented on the ZU9EG platform. The experimental results show that the proposed accelerator achieves frame rates of 509.9, 249.3, 100.7, 48.4, and 168.9 frames per second (FPS) for AlexNet, VGG-16, ResNet-18, MobileNet-v2, and EfficientNet, respectively. Compared with related works, this accelerator achieves inference speed and energy efficiency improvements of 1.1 $\times$ $\sim$ 36.1 $\times$ and 2.4 $\times$ $\sim$ 13.4 $\times$ , respectively.

OECS: A Novel Deep Convolutional Neural Network Accelerator Based on 3D Hybrid Optical-Electrical NoC

OECS: a Deep Convolutional Neural Network Accelerator Based on a 3D Hybrid Optical-Electrical NoC

A 3D Hybrid Optical-Electrical NoC Using Novel Mapping Strategy Based DCNN Dataflow Acceleration

DaDianNao: A Machine-Learning Supercomputer

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Eyelet: A Cross-Mesh NoC-Based Fine-Grained Sparse CNN Accelerator for Spatio-Temporal Parallel Computing Optimization

Neural Synaptic Plasticity-Inspired Computing: A High Computing Efficient Deep Convolutional Neural Network Accelerator

Accelerating Fully Connected Neural Network on Optical Network-on-Chip (ONoC)

An NoC-based CNN Accelerator for Edge Computing

A Parallel Loading Based Accelerator for Convolution Neural Network

Optical Convolutional Neural Network with WDM-Based Optical Patching and Microring Weighting Banks

Energy-Efficient Accelerator Design for Deformable Convolution Networks

A High Efficient Architecture for Convolution Neural Network Accelerator

Memory Bandwidth and Energy Efficiency Optimization of Deep Convolutional Neural Network Accelerators.

Design of a Convolutional Neural Network Accelerator Based on On-Chip Data Reordering

Sophisticated Deep Learning with On-Chip Optical Diffractive Tensor Processing

A 3D Tiled Low Power Accelerator for Convolutional Neural Network

Non-volatile and Ultra-Fast Photonic Vector Accelerator with Optical Phase Change Materials and Integrated Microcomb

Embedded Streaming Deep Neural Networks Accelerator With Applications

O-HAS: Optical Hardware Accelerator Search for Boosting Both Acceleration Performance and Development Speed

An Electro-Photonic System for Accelerating Deep Neural Networks