Abstract:The parallel updating scheme of RRAM-based analog neuromorphic systems based on sign stochastic gradient descent (SGD) can dramatically accelerate the training of neural networks. However, sign SGD can decrease accuracy. Also, some non-ideal factors of RRAM devices, such as intrinsic variations and the quantity of intermediate states, may significantly damage their convergence. In this paper, we analyzed the effects of these issues on the parallel updating scheme and found that it performed poorly on the task of MNIST recognition when the number of intermediate states was limited or the variation was too large. Thus, we propose a weighted synapse method to optimize the parallel updating scheme. Weighted synapses consist of major and minor synapses with different gain factors. Such a method can be widely used in RRAM-based analog neuromorphic systems to increase the number of equivalent intermediate states exponentially. The proposed method also generates a more suitable ΔW, diminishing the distortion caused by sign SGD. Unlike when several RRAM cells are combined to achieve higher resolution, there are no carry operations for weighted synapses, even if a saturation on the minor synapses occurs. The proposed method also simplifies the circuit overhead, rendering it highly suitable to the parallel updating scheme. With the aid of weighted synapses, convergence is highly optimized, and the error rate decreases significantly. Weighted synapses are also robust against the intrinsic variations of RRAM devices.

The Impact of Endurance Degradation in Analog RRAM for In-Situ Training

Characterizing Endurance Degradation of Incremental Switching in Analog RRAM for Neuromorphic Systems

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Aging Aware Retraining for Memristor-based Neuromorphic Computing

Reliable Memristor-based Neuromorphic Design Using Variation- and Defect-Aware Training

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

A Study of the Effect of RRAM Reliability Soft Errors on the Performance of RRAM-Based Neuromorphic Systems

Reliability Perspective on Neuromorphic Computing Based on Analog RRAM

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Weighted Synapses Without Carry Operations for RRAM-Based Neuromorphic Systems

A Compact Model of Analog RRAM With Device and Array Nonideal Effects for Neuromorphic Systems

Impact of Switching Window on Endurance Degradation in Analog RRAM

Compensation Architecture Design Utilizing Residual Resource to Mitigate Impacts of Nonidealities in RRAM-based Computing-in-memory Chips

Fault-Tolerant Training Enabled by On-Line Fault Detection for RRAM-Based Neural Computing Systems

Early-Stage Fluctuation in Low-Power Analog Resistive Memory: Impacts on Neural Network and Mitigation Approach

Intra-array Non-Idealities Modeling and Algorithm Optimization for RRAM-based Computing-in-Memory Applications

Endurance and Retention Degradation of Intermediate Levels in Filamentary Analog RRAM

Online Training on RRAM Based Neuromorphic Network: Experimental Demonstration and Operation Scheme Optimization

Impacts of State Instability and Retention Failure of Filamentary Analog RRAM on the Performance of Deep Neural Network

Thermal Induced Retention Degradation of RRAM-based Neuromorphic Computing Chips

Statistical Modeling of Time-Dependent Post-Programming Conductance Drift in Analog RRAM