Abstract:Emerging technologies based on resistive switching (ReRAM) devices promise to improve the speed and energy efficiency of next generation machine learning accelerators, but further research is required for achieving commercial maturity. System-level prototyping with emerging devices is costly, and algorithmic investigations require hardware neural network modeling which often deviates from experimental reality. In this work, the concept of modeling bias is proposed as a way to quantify this deviation and support reliable evaluation of device populations in the context of neural network algorithms. While applicable to other device modeling techniques, modeling bias is investigated here using jump tables - a promising physics-less technique to model emerging memory devices for hardware networks. Questions about the fidelity of these tables in relation to stochastic device behavior are answered. Two methods of jump table modeling – binning and a novel Optuna-optimized binning - are explored using synthetic data with known distributions for benchmarking and experimental data obtained from TiOx ReRAM devices for practical testing. Novel device metrics are proposed, and it is shown that these metrics can present crucial insights on the device population prior to training the hardware network. Results on a multi-layer perceptron trained on MNIST show that device models based on binning deviate from target network accuracy at a low number of points and high switching noise in the device dataset. The proposed approach opens the possibility for device-algorithm co-design investigations into statistical device models with better performance, as well as experimentally verified modeling bias in different in-memory computing and neural network architectures.

Device Variation Effects on Neural Network Inference Accuracy in Analog In‐Memory Computing Systems

The Impact of Non-linear NVM Devices on In-Memory Computing

Reliable Memristor-based Neuromorphic Design Using Variation- and Defect-Aware Training

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Mitigating RC-Delay Induced Accuracy Loss in Analog In-Memory Computing: A Non-Compromising Approach

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Compensation Architecture Design Utilizing Residual Resource to Mitigate Impacts of Nonidealities in RRAM-based Computing-in-memory Chips

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications

Exploring Cycle-to-Cycle and Device-to-Device Variation Tolerance in MLC Storage-Based Neural Network Training

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems

Accurate Program/Verify Schemes of Resistive Switching Memory (RRAM) for In-Memory Neural Network Circuits

Weighted Synapses Without Carry Operations for RRAM-Based Neuromorphic Systems

Enhanced regularization for on-chip training using analog and temporary memory weights

Suppress Variations Of Analog Resistive Memory For Neuromorphic Computing By Localizing V-O Formation

A Compact Model of Analog RRAM Considering Temperature Coefficient for Neural Network Evaluation

A Circuit-Algorithm Codesign Method to Reduce the Accuracy Drop of RRAM Based Computing-in-memory Chip

Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar.

Device Modeling Bias in ReRAM-Based Neural Network Simulations

A Variation-Aware Binary Neural Network Framework for Process Resilient In-Memory Computations

Time-dependent variability in RRAM-based analog neuromorphic system for pattern recognition