Abstract:The advances in the field of machine learning using neuromorphic systems have paved the pathway for extensive research on possibilities of hardware implementations of neural networks. Various memristive technologies such as oxide-based devices, spintronics and phase change materials have been explored to implement the core functional units of neuromorphic systems, namely the synaptic network, and the neuronal functionality, in a fast and energy efficient manner. However, various non-idealities in the crossbar implementations of the synaptic arrays can significantly degrade performance of neural networks and hence, impose restrictions on feasible crossbar sizes. In this work, we build mathematical models of various non-idealities that occur in crossbar implementations such as source resistance, neuron resistance and chip-to-chip device variations and analyze their impact on the classification accuracy of a fully connected network (FCN) and convolutional neural network (CNN) trained with standard training algorithm. We show that a network trained under ideal conditions can suffer accuracy degradation as large as 59.84% for FCNs and 62.4% for CNNs when implemented on non-ideal crossbars for relevant non-ideality ranges. This severely constrains the sizes for crossbars. As a solution, we propose a technology aware training algorithm which incorporates the mathematical models of the non-idealities in the standard training algorithm. We demonstrate that our proposed methodology achieves significant recovery of testing accuracy within 1.9% of the ideal accuracy for FCNs and 1.5% for CNNs. We further show that our proposed training algorithm can potentially allow the use of significantly larger crossbar arrays of sizes 784$\times$500 for FCNs and 4096$\times$512 for CNNs with a minor or no trade-off in accuracy

Examining the Role and Limits of Batchnorm Optimization to Mitigate Diverse Hardware-noise in In-memory Computing

The Impact of Non-linear NVM Devices on In-Memory Computing

BRoCoM: A Bayesian Framework for Robust Computing on Memristor Crossbar.

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Mitigating RC-Delay Induced Accuracy Loss in Analog In-Memory Computing: A Non-Compromising Approach

Bayesian Inference Based Robust Computing on Memristor Crossbar

Noise Injection Adaption

Training Neural Networks With In-Memory-Computing Hardware and Multi-Level Radix-4 Inputs

Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators

Technology Aware Training in Memristive Neuromorphic Systems based on non-ideal Synaptic Crossbars

Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars

Fault Tolerance in Memristive Crossbar-Based Neuromorphic Computing Systems.

Improving the accuracy of neural networks in analog computing-in-memory systems by analog weight.

Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars

Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches

ACBN: Approximate Calculated Batch Normalization for Efficient DNN On-Device Training Processor

StoX-Net: Stochastic Processing of Partial Sums for Efficient In-Memory Computing DNN Accelerators

BinSparX: Sparsified Binary Neural Networks for Reduced Hardware Non-Idealities in Xbar Arrays

Hardware/Software co-design with ADC-Less In-memory Computing Hardware for Spiking Neural Networks

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Training Physical Neural Networks for Analog In-Memory Computing