Abstract:The progress in neuromorphic computing is fueled by the development of novel nonvolatile memories capable of storing analog information and implementing neural computation efficiently. However, like most other analog circuits, these devices and circuits are prone to imperfections, such as temperature dependency, noise, tuning error, etc., often leading to considerable performance degradation in neural network implementations. Indeed, imperfections are major obstacles in the path of further progress and ultimate commercialization of these technologies. Hence, a practically viable approach should be developed to deal with these nonidealities and unleash the full potential of nonvolatile memories in neuromorphic systems. Here, for the first time, we report a comprehensive characterization of critical imperfections in two analog-grade memories, namely passively-integrated memristors and redesigned eFlash memories, which both feature long-term retention, high endurance, analog storage, low-power operation, and compact nano-scale footprint. Then, we propose a holistic approach that includes modifications in the training, tuning algorithm, memory state optimization, and circuit design to mitigate these imperfections. Our proposed methodology is corroborated on a hybrid software/experimental framework using two benchmarks: a moderate-size convolutional neural network and ResNet-18 trained on CIFAR-10 and ImageNet datasets, respectively. Our proposed approaches allow 2.5x to 9x improvements in the energy consumption of memory arrays during inference and sub-percent accuracy drop across 25-100 C temperature range. The defect tolerance is improved by >100x, and a sub-percent accuracy drop is demonstrated in deep neural networks built with 64x64 passive memristive crossbars featuring 25% normalized switching threshold variations.

Compact Modeling and Mitigation of Parasitics in Crosspoint Accelerators of Neural Networks

Modeling and Compensation of IR Drop in Crosspoint Accelerators of Neural Networks

On Designing Efficient and Reliable Nonvolatile Memory-Based Computing-In-Memory Accelerators

Modeling and Analysis of a Large-Scale Memristor Crossbar Array for Neuromorphic Chip

Reliability-Aware Training and Performance Modeling for Processing-In-Memory Systems

On the Accuracy of Analog Neural Network Inference Accelerators

Pack my weights and run! Minimizing overheads for in-memory computing accelerators

Interconnect Parasitics and Partitioning in Fully-Analog In-Memory Computing Architectures

Sparsity-Aware In-Memory Neuromorphic Computing Unit With Configurable Topology of Hybrid Spiking and Artificial Neural Network

Neural-PIM: Efficient Processing-In-Memory with Neural Approximation of Peripherals

A heterogeneous computing system with memristor-based neuromorphic accelerators

Training Physical Neural Networks for Analog In-Memory Computing

Benchmarking and modeling of analog and digital SRAM in-memory computing architectures

MNSIM 2.0: A Behavior-Level Modeling Tool for Processing-In-Memory Architectures.

Comparative Evaluation of Memory Technologies for Synaptic Crossbar Arrays -- Part I: Robustness-driven Device-Circuit Co-Design and System Implications

An Event-Based Digital Compute-In-Memory Accelerator with Flexible Operand Resolution and Layer-Wise Weight/Output Stationarity

Modeling and Signal Integrity Analysis of RRAM-Based Neuromorphic Chip Crossbar Array Using Partial Equivalent Element Circuit (PEEC) Method

Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator

Mitigating Imperfections in Mixed-Signal Neuromorphic Circuits

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems