Abstract:Since digital spiking signals can carry rich information and propagate with low computational consumption, spiking neural networks (SNNs) have received great attention from neuroscientists and are regarded as the future development object of neural networks. However, generating the appropriate spiking signals remains challenging, which is related to the dynamics property of neurons. Most existing studies imitate the biological neurons based on the correlation of synaptic input and output, but these models have only one time constant, thus ignoring the structural differentiation and versatility in biological neurons. In this article, we propose the reconstruction of adaptive leaky integrate-and-fire (R-ALIF) neuron to perform complex behaviors similar to real neurons. First, a synaptic cleft time constant is introduced into the membrane voltage charging equation to distinguish the leakage degree between the neuron membrane and the synaptic cleft, which can expand the representation space of spiking neurons to facilitate SNNs to obtain better information expression way. Second, R-ALIF constructs a voltage threshold adjustment equation to balance the firing rate of output signals. Third, three time constants are transformed into learnable parameters, enabling the adaptive adjustment of dynamics equation and enhancing the information expression ability of SNNs. Fourth, the computational graph of R-ALIF is optimized to improve the performance of SNNs. Moreover, we adopt a temporal dropout (TemDrop) method to solve the overfitting problem in SNNs and propose a data augmentation method for neuromorphic datasets. Finally, we evaluate our method on CIFAR10-DVS, ASL-DVS, and CIFAR-100, and achieve top1 accuracy of 81.0% , 99.8% , and 67.83% , respectively, with few time steps. We believe that our method will further promote the development of SNNs trained by spatiotemporal backpropagation (STBP).

Relaxation LIF: A Gradient-Based Spiking Neuron for Direct Training Deep Spiking Neural Networks

IM-LIF: Improved Neuronal Dynamics with Attention Mechanism for Direct Training Deep Spiking Neural Network

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Adaptive Deep Spiking Neural Network with Global-Local Learning Via Balanced Excitatory and Inhibitory Mechanism

Adaptive Smoothing Gradient Learning for Spiking Neural Networks.

Learnable Surrogate Gradient for Direct Training Spiking Neural Networks

Spiking Deep Residual Networks.

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

Human-Level Control Through Directly-Trained Deep Spiking Q-Networks

KLIF: An Optimized Spiking Neuron Unit for Tuning Surrogate Gradient Function

Recurrent Spiking Neural Network with Dynamic Presynaptic Currents Based on Backpropagation

Surrogate Gradient Scaling for Directly Training Spiking Neural Networks.

Memristive leaky integrate-and-fire neuron and learnable straight-through estimator in spiking neural networks

Reconstruction of Adaptive Leaky Integrate-and-Fire Neuron to Enhance the Spiking Neural Networks Performance by Establishing Complex Dynamics

Gradual Surrogate Gradient Learning in Deep Spiking Neural Networks

Deep Residual Learning in Spiking Neural Networks

Spiking Deep Residual Network

GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks

CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks