Abstract:Resistive random-access-memory (ReRAM) crossbar is a promising technique for deep neural network (DNN) accelerators, thanks to its in-memory and in-situ analog computing abilities for vector–matrix multiplication-and-accumulations (VMMs). However, it is challenging for crossbar architecture to exploit the sparsity in DNNs. It is inevitably complex and costly to exploit fine-grained sparsity due to the limitation of the tightly coupled crossbar structure. As a countermeasure, we develop a novel ReRAM-based DNN accelerator, named sparse-multiplication-engine (SME), based on a hardware and software co-design framework. First, we orchestrate the bit-sparse pattern to increase the density of bit-sparsity based on existing quantization methods. Such quantized weights can be nicely generated using the alternating direction method of multipliers (ADMM) optimization during the DNN fine-tuning, which can exactly enforce bit patterns in weights. Second, we propose a novel weight mapping mechanism to slice the bits of the weight across crossbars and splice the activation results in peripheral circuits. This mechanism can decouple the tightly coupled crossbar structure and cumulate the sparsity in the crossbar. Finally, a superior squeeze-out scheme empties the crossbars mapped with highly sparse nonzeros from the previous two steps. We design the SME architecture and discuss its use for other quantization methods and different ReRAM cell technologies. We further propose a workload grouping algorithm and a pipeline to achieve workload balance among crossbar-rows that concurrently execute multiply–accumulate operations to optimize the system latency. Putting all together, with the optimized model, compared with prior state-of-the-art designs, the SME shrinks the use of crossbars up to $8.7\times $ and $2.1\times $ using ResNet-50 and MobileNet-v2, respectively, and achieve average $3.1\times $ speed up with no or little accuracy loss on ImageNet.

Bit-Offsetter: A Bit-serial DNN Accelerator with Weight-offset MAC for Bit-wise Sparsity Exploitation

Bit-balance: Model-Hardware Co-design for Accelerating NNs by Exploiting Bit-level Sparsity

BBS: Bi-directional Bit-level Sparsity for Deep Learning Acceleration

Exploiting Bit Sparsity in Both Activation and Weight in Neural Networks Accelerators

BEM: Bit-level Sparsity-aware Deep Learning Accelerator with Efficient Booth Encoding and Weight Multiplexing

An Energy-Efficient Mixed-Bitwidth Systolic Accelerator for NAS-Optimized Deep Neural Networks

BitCluster: Fine-Grained Weight Quantization for Load-Balanced Bit-Serial Neural Network Accelerators

EBSP: evolving bit sparsity patterns for hardware-friendly inference of quantized deep neural networks

Exploiting Dynamic Bit Sparsity in Activation for Deep Neural Network Acceleration.

Elastic Significant Bit Quantization and Acceleration for Deep Neural Networks

A Precision-Scalable Deep Neural Network Accelerator with Activation Sparsity Exploitation

Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference

SoBS-X: Squeeze-Out Bit Sparsity for ReRAM-Crossbar-Based Neural Network Accelerator.

An Energy-Efficient Accelerator for Hybrid Bit-Width DNNs

A Low-Power Sparse Convolutional Neural Network Accelerator with Pre-Encoding Radix-4 Booth Multiplier

EncodingNet: A Novel Encoding-based MAC Design for Efficient Neural Network Acceleration

Bit Error Robustness for Energy-Efficient DNN Accelerators

A Precision-Scalable Energy-Efficient Bit-Split-and-Combination Vector Systolic Accelerator for NAS-Optimized DNNs on Edge

An Energy-Efficient Bit-Split-and-Combination Systolic Accelerator for NAS-Based Multi-Precision Convolution Neural Networks