Abstract:The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. However, an inevitability of transferring neural networks onto hardware is the fact that nonidealities, such as device-to-device variations or poor device yield impact performance. Methods, such as hardware-aware training, where substrate nonidealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware-based neural networks consisting of 20 000 magnetic tunnel junction (MTJ) arrays integrated on CMOS chips in a form that closely resembles scalable and market-ready spin transfer-torque magnetoresistive random access memory (STT-MRAM) technology. Using 36 dies, each containing a MTJ-CMOS crossbar array with its own nonidealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2%. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications. https://doi.org/10.1103/PhysRevApplied.21.054028 © 2024 American Physical Society

Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Aging Aware Retraining for Memristor-based Neuromorphic Computing

Bayesian Inference Based Robust Computing on Memristor Crossbar

BRoCoM: A Bayesian Framework for Robust Computing on Memristor Crossbar.

Reliable Memristor-based Neuromorphic Design Using Variation- and Defect-Aware Training

ITT-RNA: Imperfection Tolerable Training for RRAM-Crossbar-Based Deep Neural-Network Accelerator

Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar.

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

Enhance the Robustness to Time Dependent Variability of ReRAM-Based Neuromorphic Computing Systems with Regularization and 2R Synapse

Hiearchical Crossbar Design for ReRAM Based Write Variation Inhibition On-Chip Learning

Technology Aware Training in Memristive Neuromorphic Systems based on non-ideal Synaptic Crossbars

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Mitigate IR-Drop Effect by Modulating Neuron Activation Functions for Implementing Neural Networks on Memristor Crossbar Arrays

Measurement-driven neural-network training for integrated magnetic tunnel junction arrays

A Study of the Effect of RRAM Reliability Soft Errors on the Performance of RRAM-Based Neuromorphic Systems

Noise Injection Adaption

Digital Offset for RRAM-based Neuromorphic Computing: A Novel Solution to Conquer Cycle-to-cycle Variation

Online Soft Error Tolerance in ReRAM Crossbars for Deep Learning Accelerators

A Non-Idealities Aware Software–Hardware Co-Design Framework for Edge-AI Deep Neural Network Implemented on Memristive Crossbar