Abstract:As one of the most promising methods in the next generation of neuromorphic systems, memristor-based spiking neural networks (SNNs) show great advantages in terms of power efficiency, integration density, and biological plausibility. However, because of the nondifferentiability of discrete spikes, it is difficult to train SNNs with gradient descent and error backpropagation online. In this article, we propose an improved training algorithm for multilayer memristive SNN (MSNN) with three methods spontaneously, supporting in situ learning in hardware: 1) temporal order encoding is applied to generate different pulse trains in neurons; 2) a simplified homeostasis is realized by the activation state and refractory period to regulate hidden neurons spontaneously; and 3) spiking-timing-dependent plasticity (STDP) in memristive synapses is adopted to update weights in situ. Correspondingly, we provide a circuitry example and verify it in LTSPICE. Then the MSNN is benchmarked with the MNIST data set and analyzed with visualization methods, showing better recognition accuracy (95.15%) than existing SNNs with comparable scales and bio-inspired learning rules. We also consider some nonideal effects in memristor crossbar array and peripheral circuits. Evaluation results show that the proposed MSNN is robust to finite resolution, circuit noise and writing noise; and larger network scale will help the MSNN alleviate the negative impacts of other nonideal factors, including yield and device-to-device variation. Moreover, the energy efficiency of a MSNN system is estimated to achieve 7.6TOPS/W, showing great potential in low-power applications.

Expanding memory in recurrent spiking networks

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Computing of temporal information in spiking neural networks with ReRAM synapses

A bio-inspired bistable recurrent cell allows for long-lasting memory

Memory-Efficient Reversible Spiking Neural Networks

A solution to the learning dilemma for recurrent networks of spiking neurons

Learnable Surrogate Gradient for Direct Training Spiking Neural Networks

Unsupervised Learning on Resistive Memory Array Based Spiking Neural Networks

Memory and Information Processing in Recurrent Neural Networks

Incorporating structural plasticity into self-organization recurrent networks for sequence learning

Learning long sequences in spiking neural networks

In-situ Learning in Hardware Compatible Multi-layer Memristive Spiking Neural Network

Evolving Connectivity for Recurrent Spiking Neural Networks

Memory-Dependent Computation and Learning in Spiking Neural Networks Through Hebbian Plasticity

Temporal Coding in Recurrent Spiking Neural Networks with Synaptic Delay-Weight Plasticity

Spiking representation learning for associative memories

Fast learning without synaptic plasticity in spiking neural networks

Spike-based computation using classical recurrent neural networks