Abstract:Spikes are the currency in central nervous systems for information transmission and processing. They are also believed to play an essential role in low-power consumption of the biological systems, whose efficiency attracts increasing attentions to the field of neuromorphic computing. However, efficient processing and learning of discrete spikes still remain a challenging problem. In this article, we make our contributions toward this direction. A simplified spiking neuron model is first introduced with the effects of both synaptic input and firing output on the membrane potential being modeled with an impulse function. An event-driven scheme is then presented to further improve the processing efficiency. Based on the neuron model, we propose two new multispike learning rules which demonstrate better performance over other baselines on various tasks, including association, classification, and feature detection. In addition to efficiency, our learning rules demonstrate high robustness against the strong noise of different types. They can also be generalized to different spike coding schemes for the classification task, and notably, the single neuron is capable of solving multicategory classifications with our learning rules. In the feature detection task, we re-examine the ability of unsupervised spike-timing-dependent plasticity with its limitations being presented, and find a new phenomenon of losing selectivity. In contrast, our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied. Moreover, our rules cannot only detect features but also discriminate them. The improved performance of our methods would contribute to neuromorphic computing as a preferable choice.

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Learnable Surrogate Gradient for Direct Training Spiking Neural Networks

A Survey of Neuromorphic Computing Based on Spiking Neural Networks

Adaptive Smoothing Gradient Learning for Spiking Neural Networks.

Surrogate Gradient Scaling for Directly Training Spiking Neural Networks.

The Remarkable Robustness of Surrogate Gradient Learning for Instilling Complex Function in Spiking Neural Networks

Meta-learning Spiking Neural Networks with Surrogate Gradient Descent

Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networks

Directly Training Temporal Spiking Neural Network with Sparse Surrogate Gradient

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Gradual Surrogate Gradient Learning in Deep Spiking Neural Networks

Surrogate Module Learning: Reduce the Gradient Error Accumulation in Training Spiking Neural Networks

Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks.

Randomized Forward Mode Gradient for Spiking Neural Networks in Scientific Machine Learning

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Efficient Surrogate Gradients for Training Spiking Neural Networks

Improving Surrogate Gradient Learning in Spiking Neural Networks via Regularization and Normalization

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting

Towards Efficient On-Chip Learning for Spiking Neural Networks Accelerator with Surrogate Gradient

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology