Abstract:Deep neural networks (DNNs) have gained a strong momentum among various applications. The enormous matrix-multiplication exhibited in the above DNNs is computation and memory intensive. Resistive random-access memory crossbar (RRAM-crossbar) consisting of memristor cells can naturally carry out the matrix-vector multiplication. RRAM-crossbar-based accelerator, therefore, has two orders of magnitude of higher energy-efficiency than conventional accelerators. The imperfect fabrication process of RRAM-crossbars, however, causes various defects and process variations. These fabrication imperfections not only result in significant yield loss but also degrade the accuracy of DNNs executed on the RRAM-crossbars. In this article, we first propose an accelerator-friendly neural-network training method, by leveraging the inherent self-healing capability of the neural network, to prevent the large-weight synapses from being mapped to the imperfect memristors. Next, we propose a dynamic adjustment mechanism to extend the above method for DNNs, such as multilayer perceptrons (MLPs), wherein the imperfect-memristor induced errors can accumulate and magnify through multiple layers. Such off-device training method is a pure software solution, and it is unable to provide enough accuracy for convolutional neural networks (CNNs). Several works propose error-tolerable hardware design by allowing the retraining of CNNs on the RRAM-crossbar. Although this hardware-based on-device training method is effective, the frequent write operation on RRAM-crossbar hurt the endurance of RRAM-crossbars. Consequently, we propose a software and hardware co-design methodology to effectively preserve the classification accuracy of CNN with few on-device training iterations. The experimental results show that the proposed method can guarantee ≤1.1% loss of accuracy for resistance variations in MLP and CNN. Moreover, the proposed method can guarantee ≤1% loss of accuracy even when stuck-at-faults (SAFs) rate = 20%.

Exploring the Impact of Random Telegraph Noise-Induced Accuracy Loss on Resistive RAM-Based Deep Neural Network

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

RTN Impacts on RRAM-based Nonvolatile Logic Circuit

Early-Stage Fluctuation in Low-Power Analog Resistive Memory: Impacts on Neural Network and Mitigation Approach

Drop-Connect as a Fault-Tolerance Approach for RRAM-based Deep Neural Network Accelerators

Impact of RTN and Variability on RRAM-Based Neural Network

Evaluation of SRAM V-min Shift Induced by Random Telegraph Noise (RTN): Physical Understanding and Prediction Method

ITT-RNA: Imperfection Tolerable Training for RRAM-Crossbar-Based Deep Neural-Network Accelerator

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Impacts of State Instability and Retention Failure of Filamentary Analog RRAM on the Performance of Deep Neural Network

Time-dependent variability in RRAM-based analog neuromorphic system for pattern recognition

A Study of the Effect of RRAM Reliability Soft Errors on the Performance of RRAM-Based Neuromorphic Systems

Rescuing RRAM-Based Computing from Static and Dynamic Faults

Noise Injection Adaption

Fast and Accurate Weight Updating Strategy for Resistive Random-Access Memory (Rram)-Based Neural Networks

Improving DNN Fault Tolerance Using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI

Digital Offset for RRAM-based Neuromorphic Computing: A Novel Solution to Conquer Cycle-to-cycle Variation

Training Low Bitwidth Convolutional Neural Network on RRAM

Spiking Neural Network with RRAM: Can We Use It for Real-World Application?

Evaluating Neural Network-Inspired Analog-to-Digital Conversion with Low-Precision RRAM

TIME: A Training-in-Memory Architecture for RRAM-Based Deep Neural Networks