Abstract:In the real world, information is often correlated with each other in the time domain. Whether it can effectively make a decision according to the global information is the key indicator of information processing ability. Due to the discrete characteristics of spike trains and unique temporal dynamics, spiking neural networks (SNNs) show great potential in applications in ultra-low-power platforms and various temporal-related real-life tasks. However, the current SNNs can only focus on the information a short time before the current moment, its sensitivity in the time domain is limited. This problem affects the processing ability of SNN in different kinds of data, including static data and time-variant data, and reduces the application scenarios and scalability of SNN. In this work, we analyze the impact of such information loss and then integrate SNN with working memory inspired by recent neuroscience research. Specifically, we propose Spiking Neural Networks with Working Memory (SNNWM) to handle input spike trains segment by segment. On the one hand, this model can effectively increase SNN's ability to obtain global information. On the other hand, it can effectively reduce the information redundancy between adjacent time steps. Then, we provide simple methods to implement the proposed network architecture from the perspectives of biological plausibility and neuromorphic hardware friendly. Finally, we test the proposed method on static and sequential data sets, and the experimental results show that the proposed model can better process the whole spike train, and achieve state-of-the-art results in short time steps. This work investigates the contribution of introducing biologically inspired mechanisms, e.g., working memory, and multiple delayed synapses to SNNs, and provides a new perspective to design future SNNs.

Unleashing the Potential of Spiking Neural Networks for Sequential Modeling with Contextual Embedding.

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Context Gating in Spiking Neural Networks: Achieving Lifelong Learning through Integration of Local and Global Plasticity

TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling

A Spike-Timing Based Integrated Model for Pattern Recognition

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

A Spiking Neural Network Model For Associative Memory Using Temporal Codes

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

A Spiking Neural Network System for Robust Sequence Recognition

An end-to-end functional spiking model for sequential feature learning

Sparse Temporal Encoding of Visual Features for Robust Object Recognition by Spiking Neurons

A Hierarchical Organized Memory Model Using Spiking Neurons

Spiking Deep Residual Networks.

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Similarity-based context aware continual learning for spiking neural networks

Spiking neural network with working memory can integrate and rectify spatiotemporal features

Efficient Online Learning for Networks of Two-Compartment Spiking Neurons

Learning Long Sequences in Spiking Neural Networks