Abstract:Emerging technologies based on resistive switching (ReRAM) devices promise to improve the speed and energy efficiency of next generation machine learning accelerators, but further research is required for achieving commercial maturity. System-level prototyping with emerging devices is costly, and algorithmic investigations require hardware neural network modeling which often deviates from experimental reality. In this work, the concept of modeling bias is proposed as a way to quantify this deviation and support reliable evaluation of device populations in the context of neural network algorithms. While applicable to other device modeling techniques, modeling bias is investigated here using jump tables - a promising physics-less technique to model emerging memory devices for hardware networks. Questions about the fidelity of these tables in relation to stochastic device behavior are answered. Two methods of jump table modeling – binning and a novel Optuna-optimized binning - are explored using synthetic data with known distributions for benchmarking and experimental data obtained from TiOx ReRAM devices for practical testing. Novel device metrics are proposed, and it is shown that these metrics can present crucial insights on the device population prior to training the hardware network. Results on a multi-layer perceptron trained on MNIST show that device models based on binning deviate from target network accuracy at a low number of points and high switching noise in the device dataset. The proposed approach opens the possibility for device-algorithm co-design investigations into statistical device models with better performance, as well as experimentally verified modeling bias in different in-memory computing and neural network architectures.

Modeling and Analysis of Analog Non-Volatile Devices for Compute-In-Memory Applications

The Impact of Non-linear NVM Devices on In-Memory Computing

Integration of Physics-Derived Memristor Models with Machine Learning Frameworks

Analysis of Dynamic Linear and Non-linear Memristor Device Models for Emerging Neuromorphic Computing Hardware Design

MNSIM-TIME: Performance Modeling Framework for Training-In-Memory Architectures

CIMulator: A Comprehensive Simulation Platform for Computing-In-Memory Circuit Macros with Low Bit-Width and Real Memory Materials

Latency Insertion Method for Accelerated Simulation of Memristor Crossbar Array in Neuromorphic Chip

Modeling and Analysis of a Large-Scale Memristor Crossbar Array for Neuromorphic Chip

Analysis and Modeling of Internal State Variables for Dynamic Effects of Nonvolatile Memory Devices

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems

Device Modeling Bias in ReRAM-Based Neural Network Simulations

MNSIM 2.0: A Behavior-Level Modeling Tool for Memristor-based Neuromorphic Computing Systems

A Co-design view of Compute in-Memory with Non-Volatile Elements for Neural Networks

Training Physical Neural Networks for Analog In-Memory Computing

A flexible and fast digital twin for RRAM systems applied for training resilient neural networks

DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Training

MNSIM: Simulation Platform for Memristor-Based Neuromorphic Computing System.

MNSIM 2.0: A Behavior-Level Modeling Tool for Processing-In-Memory Architectures.

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications

Benchmarking and modeling of analog and digital SRAM in-memory computing architectures