Abstract:The emerging resistive random-access memory (RRAM) has been widely applied in accelerating the computing of deep neural networks. However, it is challenging to achieve high-precision computations based on RRAM due to the limits of the resistance level and the interfaces. Low bit-width convolutional neural networks (CNNs) provide promising solutions to introduce low bit-width RRAM devices and low bit-width interfaces in RRAM-based computing system (RCS). While open questions still remain regarding: 1) how to make matrix splitting when a single crossbar is not large enough to hold all parameters of one weight matrix; 2) how to design a pipeline to accelerate the inference based on line buffer structure; and 3) how to reduce the accuracy drop due to the parameter splitting and data quantization. In this paper, we propose an RRAM crossbar-based low bit-width CNN (LB-CNN) accelerator. We make detailed discussion on the system design, including the matrix splitting strategies to enhance the scalability, and the pipelined implementation based on line buffers to accelerate the inference. In addition, we propose a splitting and quantizing while training method to incorporate the actual hardware constraints with the training. In our experiments, low bit-width LeNet-5 on RRAM show much better robustness than multibit models with device variation. The pipeline strategy achieves approximately 6.0x speedup to process each image on ResNet-18. For low-bit VGG-8 on CIFAR-10, the proposed accelerator saves 54.9% of the energy consumption and 48.3% of the area compared with the multibit VGG-8 structure.

CNNWire

An Energy-Efficient Mixed-Bit CNN Accelerator With Column Parallel Readout for ReRAM-Based In-Memory Computing

A 1T2R1C ReRAM CIM Accelerator with Energy-Efficient Voltage Division and Capacitive Coupling for CNN Acceleration in AI Edge Applications.

Efficient Implementation of Multi-Channel Convolution in Monolithic 3D ReRAM Crossbar

NAND-SPIN-based processing-in-MRAM architecture for convolutional neural network acceleration

ReHy: A ReRAM-based Digital/Analog Hybrid PIM Architecture for Accelerating CNN Training

3A-Reram: Adaptive Activation Accumulation in ReRAM-Based CNN Accelerator

XB-SIM∗: A Simulation Framework for Modeling and Exploration of ReRAM-based CNN Acceleration Design

Training Low Bitwidth Convolutional Neural Network on RRAM

Low Bit-Width Convolutional Neural Network on RRAM

High Area/Energy Efficiency RRAM CNN Accelerator with Pattern-Pruning-Based Weight Mapping Scheme

Re2PIM

A RRAM Based Max-Pooling Scheme for Convolutional Neural Network

Accelerating Low Bit-Width Deep Convolution Neural Network in MRAM.

RRAM Based Buffer Design for Energy Efficient CNN Accelerator.

A 3.89-Gops/mw Scalable Recurrent Neural Network Processor with Improved Efficiency on Memory and Computation

ReCom: an Efficient Resistive Accelerator for Compressed Deep Neural Networks.

Ultra-High-Speed Accelerator Architecture for Convolutional Neural Network Based on Processing-in-Memory Using Resistive Random Access Memory

Hubpa: High Utilization Bidirectional Pipeline Architecture For Neuromorphic Computing

A Novel Low-Communication Energy-Efficient Reconfigurable CNN Acceleration Architecture

A Configurable Multi-Precision CNN Computing Framework Based on Single Bit RRAM