Enhanced Spiking Neural Network with forgetting phenomenon based on electronic synaptic devices

Jiwei Li,Hui Xu,Sheng-Yang Sun,Sen Liu,Nan Li,Qingjiang Li,Haijun Liu,Zhiwei Li

DOI: https://doi.org/10.1016/j.neucom.2019.09.030

IF: 6

2020-09-01

Neurocomputing

Abstract:Forgetting is an essential phenomenon in human brain to help people get away from chaos. Such phenomenon could be emulated in the electronic synaptic device, which is the critical unit for hardware implementation of Spiking Neural Network (SNN). To the best of our knowledge, however, forgetting phenomenon has rarely been applied in weights update of traditional SNNs based on spike-timing-dependent plasticity (STDP). In this work, we propose a novel SNN training algorithm using forgetting phenomenon. The weights update procedures are composed of potentiation and forgetting, and implemented by single-polarity pulses and time intervals between training samples. Benchmarked with the MNIST handwriting dataset, we demonstrate the algorithm’s performance by a single-layer perceptron with 784  ×  10 synapses. Besides, the influence of some non-ideal factors are taken into consideration to analyze the robust performance of enhanced SNN. The simulation result indicates the proposed SNN with forgetting phenomenon exhibits faster convergence speed and higher recognition rate (88.07%) than traditional SNN with similar network scale. Moreover, it shows a good tolerance to the non-linear conductance response and variation while it has less endurance requirement of electronic synaptic devices.

computer science, artificial intelligence

What problem does this paper attempt to address?

The paper mainly aims to address the following issues: 1. **Introducing the forgetting phenomenon into artificial synaptic devices:** The paper proposes a new Spiking Neural Network (SNN) training algorithm that utilizes the forgetting phenomenon in electronic synaptic devices to update weights. This forgetting phenomenon can simplify circuit implementation and mitigate the sneak path problem. 2. **Improving SNN performance:** By incorporating the forgetting mechanism during the weight update process, the goal is to enhance the convergence speed and recognition rate of SNNs. Experimental results show that the new method achieves faster convergence speed and higher recognition rate (up to 88.07%) on a single-layer perceptron compared to traditional SNNs. 3. **Handling the impact of non-ideal factors:** The paper considers some non-ideal factors in actual electronic synaptic devices, such as nonlinear conductance response and variations in forgetting characteristics, and analyzes their impact on the performance of the enhanced SNN. The results show that the proposed SNN has better tolerance to nonlinear conductance response and is suitable for electronic synaptic devices with lower durability. 4. **Simplifying hardware implementation:** The new training algorithm simplifies hardware implementation because it does not require the application of negative polarity pulses for the weight decrement process, thereby simplifying circuit design and alleviating the sneak current problem. In summary, the main contribution of this paper is the proposal of a new SNN training algorithm that incorporates the forgetting phenomenon. This algorithm not only improves the performance of SNNs but also takes into account non-ideal factors in practical applications, providing an effective solution for the implementation of brain-like artificial intelligence systems based on electronic synaptic devices.

Enhanced Spiking Neural Network with forgetting phenomenon based on electronic synaptic devices

ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks

In-situ Learning in Hardware Compatible Multi-layer Memristive Spiking Neural Network

Binary Resistive-Switching-Device-Based Electronic Synapse with Spike-Rate-Dependent Plasticity for Online Learning

A Memristor-Based Spiking Neural Network With High Scalability and Learning Efficiency

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks

A Low-Cost Hardware-Friendly Spiking Neural Network Based on Binary MRAM Synapses, Accelerated Using In-Memory Computing

SNAP: Stopping Catastrophic Forgetting in Hebbian Learning with Sigmoidal Neuronal Adaptive Plasticity

Forgetting memristor based STDP learning circuit for neural networks

A brain-inspired algorithm that mitigates catastrophic forgetting of artificial and spiking neural networks with low computational cost

SSTDP: Supervised Spike Timing Dependent Plasticity for Efficient Spiking Neural Network Training

An Energy Efficient STDP-Based SNN Architecture With On-Chip Learning

Experimental Demonstration of Supervised Learning in Spiking Neural Networks with Phase-Change Memory Synapses

A 4096-Neuron 1M-Synapse 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS

High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning

Stochastic spin-orbit-torque device as the STDP synapse for spiking neural networks

A Digitalized RRAM-based Spiking Neuron Network System with 3-Bit Weight and Unsupervised Online Learning Scheme

In-situ learning in multilayer locally-connected memristive spiking neural network

A neural network circuit with associative learning and forgetting process based on memristor neuromorphic device

Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness