Abstract:The parallel updating scheme of RRAM-based analog neuromorphic systems based on sign stochastic gradient descent (SGD) can dramatically accelerate the training of neural networks. However, sign SGD can decrease accuracy. Also, some non-ideal factors of RRAM devices, such as intrinsic variations and the quantity of intermediate states, may significantly damage their convergence. In this paper, we analyzed the effects of these issues on the parallel updating scheme and found that it performed poorly on the task of MNIST recognition when the number of intermediate states was limited or the variation was too large. Thus, we propose a weighted synapse method to optimize the parallel updating scheme. Weighted synapses consist of major and minor synapses with different gain factors. Such a method can be widely used in RRAM-based analog neuromorphic systems to increase the number of equivalent intermediate states exponentially. The proposed method also generates a more suitable ΔW, diminishing the distortion caused by sign SGD. Unlike when several RRAM cells are combined to achieve higher resolution, there are no carry operations for weighted synapses, even if a saturation on the minor synapses occurs. The proposed method also simplifies the circuit overhead, rendering it highly suitable to the parallel updating scheme. With the aid of weighted synapses, convergence is highly optimized, and the error rate decreases significantly. Weighted synapses are also robust against the intrinsic variations of RRAM devices.

Optimized learning scheme for grayscale image recognition in a RRAM based analog neuromorphic system

Robustness of Neuromorphic Computing with RRAM-based Crossbars and Optical Neural Networks.

Device and circuit optimization of RRAM for neuromorphic computing

Design and Hardware Implementation of Neuromorphic Systems with RRAM Synapses and Threshold-Controlled Neurons for Pattern Recognition.

A neuromorphic visual system using RRAM synaptic devices with Sub-pJ energy and tolerance to variability: Experimental characterization and large-scale modeling

Sign backpropagation: An on-chip learning algorithm for analog RRAM neuromorphic computing systems

Circuit Design of RRAM-based Neuromorphic Hardware Systems for Classification and Modified Hebbian Learning

Weighted Synapses Without Carry Operations for RRAM-Based Neuromorphic Systems

Optimized Programming Scheme Enabling Symmetric Conductance Modulation in HfO₂ Resistive Random-Access Memory (RRAM) for Neuromorphic Systems

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

Online Training on RRAM Based Neuromorphic Network: Experimental Demonstration and Operation Scheme Optimization

A Digitalized RRAM-based Spiking Neuron Network System with 3-Bit Weight and Unsupervised Online Learning Scheme

An Energy-Efficient Computing-in-Memory Neuromorphic System with On-Chip Training.

Optoelectronic resistive random access memory for neuromorphic vision sensors

RRAM based synaptic devices for neuromorphic visual systems

Characterizing Endurance Degradation of Incremental Switching in Analog RRAM for Neuromorphic Systems

Neuromorphic Computing Based on Resistive RAM.

Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron

A Spike Neuron Network Prototype Based on RRAM Array

Time-dependent variability in RRAM-based analog neuromorphic system for pattern recognition