Abstract:The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as device-to-device variations or poor device yield impact performance. Methods such as hardware-aware training, where substrate non-idealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware neural networks consisting of 20,000 magnetic tunnel junction arrays integrated on a complementary metal-oxide-semiconductor chips that closely resembles market-ready spin transfer-torque magnetoresistive random access memory technology. Using 36 dies, each containing a crossbar array with its own non-idealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2 %. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.

Magnetoresistive RAM for error resilient XNOR-Nets

Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Reliable Memristor-based Neuromorphic Design Using Variation- and Defect-Aware Training

SOT-MRAM-Based Design for Energy-Efficient and Reliable Binary Neural Network Acceleration

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

Towards Explainable Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks

Bit-Aware Fault-Tolerant Hybrid Retraining and Remapping Schemes for RRAM-Based Computing-in-Memory Systems

Online Soft Error Tolerance in ReRAM Crossbars for Deep Learning Accelerators

SOT-MRAM-Enabled Probabilistic Binary Neural Networks for Noise-Tolerant and Fast Training

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Error Correction Scheme for Reliable RRAM-Based In-Memory Computing

A Low-Cost Hardware-Friendly Spiking Neural Network Based on Binary MRAM Synapses, Accelerated Using In-Memory Computing

Binary Neural Network with 16 Mb Rram Macro Chip for Classification and Online Training

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications

Measurement-driven neural-network training for integrated magnetic tunnel junction arrays

Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars

Training Low Bitwidth Convolutional Neural Network on RRAM

Robustness of Neural Networks against Storage Media Errors

A Variation-Aware Binary Neural Network Framework for Process Resilient In-Memory Computations

Binary Convolutional Neural Network on RRAM.