Abstract:Spikes are the currency in central nervous systems for information transmission and processing. They are also believed to play an essential role in low-power consumption of the biological systems, whose efficiency attracts increasing attentions to the field of neuromorphic computing. However, efficient processing and learning of discrete spikes still remain a challenging problem. In this article, we make our contributions toward this direction. A simplified spiking neuron model is first introduced with the effects of both synaptic input and firing output on the membrane potential being modeled with an impulse function. An event-driven scheme is then presented to further improve the processing efficiency. Based on the neuron model, we propose two new multispike learning rules which demonstrate better performance over other baselines on various tasks, including association, classification, and feature detection. In addition to efficiency, our learning rules demonstrate high robustness against the strong noise of different types. They can also be generalized to different spike coding schemes for the classification task, and notably, the single neuron is capable of solving multicategory classifications with our learning rules. In the feature detection task, we re-examine the ability of unsupervised spike-timing-dependent plasticity with its limitations being presented, and find a new phenomenon of losing selectivity. In contrast, our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied. Moreover, our rules cannot only detect features but also discriminate them. The improved performance of our methods would contribute to neuromorphic computing as a preferable choice.

Smooth Exact Gradient Descent Learning in Spiking Neural Networks

Adaptive Smoothing Gradient Learning for Spiking Neural Networks.

Gradient Descent for Spiking Neural Networks

Learnable Surrogate Gradient for Direct Training Spiking Neural Networks

Event-based backpropagation can compute exact gradients for spiking neural networks

Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks.

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Randomized Forward Mode Gradient for Spiking Neural Networks in Scientific Machine Learning

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

An Approximate Gradient Descent Algorithm for Spiking Neural Network

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

SpikePropamine: Differentiable Plasticity in Spiking Neural Networks

Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networks

High-performance deep spiking neural networks with 0.3 spikes per neuron

Natural-gradient learning for spiking neurons

Gradient-based methods for spiking physical systems

Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks

Gradual Surrogate Gradient Learning in Deep Spiking Neural Networks

A solution to the learning dilemma for recurrent networks of spiking neurons