Abstract:Deep neural networks (DNNs) have gained a strong momentum among various applications. The enormous matrix-multiplication exhibited in the above DNNs is computation and memory intensive. Resistive random-access memory crossbar (RRAM-crossbar) consisting of memristor cells can naturally carry out the matrix-vector multiplication. RRAM-crossbar-based accelerator, therefore, has two orders of magnitude of higher energy-efficiency than conventional accelerators. The imperfect fabrication process of RRAM-crossbars, however, causes various defects and process variations. These fabrication imperfections not only result in significant yield loss but also degrade the accuracy of DNNs executed on the RRAM-crossbars. In this article, we first propose an accelerator-friendly neural-network training method, by leveraging the inherent self-healing capability of the neural network, to prevent the large-weight synapses from being mapped to the imperfect memristors. Next, we propose a dynamic adjustment mechanism to extend the above method for DNNs, such as multilayer perceptrons (MLPs), wherein the imperfect-memristor induced errors can accumulate and magnify through multiple layers. Such off-device training method is a pure software solution, and it is unable to provide enough accuracy for convolutional neural networks (CNNs). Several works propose error-tolerable hardware design by allowing the retraining of CNNs on the RRAM-crossbar. Although this hardware-based on-device training method is effective, the frequent write operation on RRAM-crossbar hurt the endurance of RRAM-crossbars. Consequently, we propose a software and hardware co-design methodology to effectively preserve the classification accuracy of CNN with few on-device training iterations. The experimental results show that the proposed method can guarantee ≤1.1% loss of accuracy for resistance variations in MLP and CNN. Moreover, the proposed method can guarantee ≤1% loss of accuracy even when stuck-at-faults (SAFs) rate = 20%.

Low Bit-Width Convolutional Neural Network on RRAM

Training Low Bitwidth Convolutional Neural Network on RRAM

Binary Convolutional Neural Network on RRAM.

A Configurable Multi-Precision CNN Computing Framework Based on Single Bit RRAM

Convolutional Neural Networks Based on RRAM Devices for Image Recognition and Online Learning Tasks

Accelerating Low Bit-Width Deep Convolution Neural Network in MRAM.

Switched by input: power efficient structure for RRAM-based convolutional neural network.

SNrram: an Efficient Sparse Neural Network Computation Architecture Based on Resistive Random-Access Memory.

A High-Throughput and Energy-Efficient RRAM-based Convolutional Neural Network Using Data Encoding and Dynamic Quantization.

CompRRAE: RRAM-based Convolutional Neural Network Accelerator with Reduced Computations through a Runtime Activation Estimation

An On-chip Layer-wise Training Method for RRAM Based Computing-in-memory Chips.

RRAM Based Convolutional Neural Networks for High Accuracy Pattern Recognition and Online Learning Tasks

Hardware Implementation Of Rram Based Binarized Neural Networks

An Energy-Efficient Mixed-Bit CNN Accelerator With Column Parallel Readout for ReRAM-Based In-Memory Computing

RRAM-DNN: an RRAM and Model-Compression Empowered All-Weights-On-Chip DNN Accelerator

ITT-RNA: Imperfection Tolerable Training for RRAM-Crossbar-Based Deep Neural-Network Accelerator

A RRAM Based Max-Pooling Scheme for Convolutional Neural Network

AEPE: an Area and Power Efficient RRAM Crossbar-Based Accelerator for Deep CNNs

RRAM Based Buffer Design for Energy Efficient CNN Accelerator.

High Area/Energy Efficiency RRAM CNN Accelerator with Pattern-Pruning-Based Weight Mapping Scheme

Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators