Abstract:Recent advances in deep neural networks (DNNs) have shown Binary Neural Networks (BNNs) are able to provide a reasonable accuracy on various image datasets with a significant reduction in computation and memory cost. In this paper, we explore two BNNs: hybrid BNN (HBNN) and XNOR-BNN, where the weights are binarized to +1/-1 while the neuron activations are binarized to 1/0 and +1/-1, respectively. Two SRAM bit cell designs are proposed, namely, 6T SRAM for HBNN and customized 8T SRAM for XNOR-BNN. In our design, the high-precision multiply-and-accumulate (MAC) is replaced by bitwise multiplication for HBNN or XNOR for XNOR-BNN plus bit-counting operations. To parallelize the weighted sum operation, we activate multiple word lines in the SRAM array simultaneously and digitize the analog voltage developed along the bit line by a multi-level sense amplifier (MLSA). In order to partition the large matrices in DNNs, we investigate the impact of sensing bit-levels of MLSA on the accuracy degradation for different sub-array sizes and propose using the nonlinear quantization technique to mitigate the accuracy degradation. With 64 × 64 sub-array size and 3-bit MLSA, HBNN and XNOR-BNN architectures can minimize the accuracy degradation to 2.37% and 0.88%, respectively, for an inspired VGG-16 network on the CIFAR-10 dataset. Design space exploration of SRAM based synaptic architectures with the conventional row-by-row access scheme and our proposed parallel access scheme are also performed, showing significant benefits in the area, latency and energy-efficiency. Finally, we have successfully taped-out and validated the proposed HBNN and XNOR-BNN designs in TSMC 65 nm process with measured silicon data, achieving energy-efficiency >100 TOPS/W for HBNN and >50 TOPS/W for XNOR-BNN.

Hardware Implementation Of Rram Based Binarized Neural Networks

An Improved RRAM-Based Binarized Neural Network with High Variation-Tolerated Forward/Backward Propagation Module

Rram-Based Binary Neural Networks Using Back-Propagation Learning

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

An Improved Hardware Accelaration Architecture of Binary Neural Network With 1T1R Array Based Forward/Backward Propagation Module

Device and circuit optimization of RRAM for neuromorphic computing

Fast and Accurate Weight Updating Strategy for Resistive Random-Access Memory (Rram)-Based Neural Networks

Sign backpropagation: An on-chip learning algorithm for analog RRAM neuromorphic computing systems

Convolutional Neural Networks Based on RRAM Devices for Image Recognition and Online Learning Tasks

Weighted Synapses Without Carry Operations for RRAM-Based Neuromorphic Systems

Parallelizing SRAM Arrays with Customized Bit-Cell for Binary Neural Networks

Low Bit-Width Convolutional Neural Network on RRAM

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems

Circuit Design of RRAM-based Neuromorphic Hardware Systems for Classification and Modified Hebbian Learning

Neuromorphic Computing Based on Resistive RAM.

High-Throughput In-Memory Computing for Binary Deep Neural Networks with Monolithically Integrated RRAM and 90nm CMOS

Optimized Programming Scheme Enabling Symmetric Conductance Modulation in HfO₂ Resistive Random-Access Memory (RRAM) for Neuromorphic Systems

Training Low Bitwidth Convolutional Neural Network on RRAM

Area Efficient Pattern Representation of Binary Neural Networks on RRAM

Binary Convolutional Neural Network on RRAM.

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications