Abstract:The parallel updating scheme of RRAM-based analog neuromorphic systems based on sign stochastic gradient descent (SGD) can dramatically accelerate the training of neural networks. However, sign SGD can decrease accuracy. Also, some non-ideal factors of RRAM devices, such as intrinsic variations and the quantity of intermediate states, may significantly damage their convergence. In this paper, we analyzed the effects of these issues on the parallel updating scheme and found that it performed poorly on the task of MNIST recognition when the number of intermediate states was limited or the variation was too large. Thus, we propose a weighted synapse method to optimize the parallel updating scheme. Weighted synapses consist of major and minor synapses with different gain factors. Such a method can be widely used in RRAM-based analog neuromorphic systems to increase the number of equivalent intermediate states exponentially. The proposed method also generates a more suitable ΔW, diminishing the distortion caused by sign SGD. Unlike when several RRAM cells are combined to achieve higher resolution, there are no carry operations for weighted synapses, even if a saturation on the minor synapses occurs. The proposed method also simplifies the circuit overhead, rendering it highly suitable to the parallel updating scheme. With the aid of weighted synapses, convergence is highly optimized, and the error rate decreases significantly. Weighted synapses are also robust against the intrinsic variations of RRAM devices.

Adaptive Weight Mapping Strategy to Address the Parasitic Effects for ReRAM-based Neural Networks

Improving DNN Fault Tolerance Using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI

ReMap: Reorder Mapping for Multi-level Uneven Distribution on Sparse ReRAM Accelerator.

Bit-Aware Fault-Tolerant Hybrid Retraining and Remapping Schemes for RRAM-Based Computing-in-Memory Systems

Fast and Accurate Weight Updating Strategy for Resistive Random-Access Memory (Rram)-Based Neural Networks

Circuit Modeling for RRAM-Based Neuromorphic Chip Crossbar Array with and Without Write-Verify Scheme.

Compensation Architecture to Alleviate Noise Effects in RRAM-based Computing-in-memory Chips with Residual Resource

Learning the sparsity for ReRAM - mapping and pruning sparse neural network for ReRAM based accelerator.

Compensation Architecture Design Utilizing Residual Resource to Mitigate Impacts of Nonidealities in RRAM-based Computing-in-memory Chips

Learning the Sparsity for ReRAM

Weighted Synapses Without Carry Operations for RRAM-Based Neuromorphic Systems

Unary Coding and Variation-Aware Optimal Mapping Scheme for Reliable ReRAM-Based Neuromorphic Computing.

Area Efficient Pattern Representation of Binary Neural Networks on RRAM

ExtendLife: Weights Mapping Framework to Improve RRAM Lifetime for Accelerating CNN.

ESNreram: an Energy-Efficient Sparse Neural Network Based on Resistive Random-Access Memory.

PRAP-PIM: A Weight Pattern Reusing Aware Pruning Method for ReRAM-based PIM DNN Accelerators

Go Unary: A Novel Synapse Coding And Mapping Scheme For Reliable Reram-Based Neuromorphic Computing

Digital Offset for RRAM-based Neuromorphic Computing: A Novel Solution to Conquer Cycle-to-cycle Variation

Cross-layer Designs against Non-ideal Effects in ReRAM-based Processing-in-Memory System

Write or Not: Programming Scheme Optimization for RRAM-based Neuromorphic Computing

Efficient and Robust RRAM-Based Convolutional Weight Mapping With Shifted and Duplicated Kernel