Abstract:The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. However, an inevitability of transferring neural networks onto hardware is the fact that nonidealities, such as device-to-device variations or poor device yield impact performance. Methods, such as hardware-aware training, where substrate nonidealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware-based neural networks consisting of 20 000 magnetic tunnel junction (MTJ) arrays integrated on CMOS chips in a form that closely resembles scalable and market-ready spin transfer-torque magnetoresistive random access memory (STT-MRAM) technology. Using 36 dies, each containing a MTJ-CMOS crossbar array with its own nonidealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2%. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications. https://doi.org/10.1103/PhysRevApplied.21.054028 © 2024 American Physical Society

Effect of stochastic activation function on reconstruction performance of restricted Boltzmann machines with stochastic magnetic tunnel junctions

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Stochastic Spiking Neural Networks Enabled by Magnetic Tunnel Junctions: From Nontelegraphic to Telegraphic Switching Regimes

Intrinsic Spike Timing Dependent Plasticity in Stochastic Magnetic Tunnel Junctions Mediated by Heat Dynamics

Design Space Exploration of Magnetic Tunnel Junction based Stochastic Computing in Deep Learning.

Neuromorphic Hebbian learning with magnetic tunnel junction synapses

Restricted Boltzmann Machines Implemented by Spin-Orbit Torque Magnetic Tunnel Junctions

Stochastic Domain Wall-Magnetic Tunnel Junction Artificial Neurons for Noise-Resilient Spiking Neural Networks

Decision Making Module Based on Stochastic Magnetic Tunnel Junctions

Stochastic Artificial Synapses Based on Nanoscale Magnetic Tunnel Junction for Neuromorphic Applications

Stochastic IMT (insulator-metal-transition) neurons: An interplay of thermal and threshold noise at bifurcation

Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning

Reconfigurable Stochastic neurons based on tin oxide/MoS2 hetero-memristors for simulated annealing and the Boltzmann machine

A novel compound synapse using probabilistic spin-orbit-torque switching for MTJ based deep neural networks

Bi-sigmoid spike-timing dependent plasticity learning rule for magnetic tunnel junction-based SNN

Mott memristor based stochastic neurons for probabilistic computing

Measurement-driven neural-network training for integrated magnetic tunnel junction arrays

Machine learning using magnetic stochastic synapses

Stochastic Spintronic Device Based Synapses and Spiking Neurons for Neuromorphic Computation

Realization of Neural Coding by Stochastic Switching of Magnetic Tunnel Junction.

Neuroscience-inspired Information-Integration System Based on Stochastic Magnetic Tunnel Junctions