Abstract:Efficient implementations of spiking neural networks on neuromorphic hardware promise orders of magnitude less power consumption than their non-spiking counterparts. The standard neuron model for spike-based computation on such neuromorphic systems has long been the leaky integrate-and-fire (LIF) neuron. As a promising advancement, a computationally light augmentation of the LIF neuron model with an adaptation mechanism experienced a recent upswing in popularity, caused by demonstrations of its superior performance on spatio-temporal processing tasks. The root of the superiority of these so-called adaptive LIF neurons however, is not well understood. In this article, we thoroughly analyze the dynamical, computational, and learning properties of adaptive LIF neurons and networks thereof. We find that the frequently observed stability problems during training of such networks can be overcome by applying an alternative discretization method that results in provably better stability properties than the commonly used Euler-Forward method. With this discretization, we achieved a new state-of-the-art performance on common event-based benchmark datasets. We also show that the superiority of networks of adaptive LIF neurons extends to the prediction and generation of complex time series. Our further analysis of the computational properties of networks of adaptive LIF neurons shows that they are particularly well suited to exploit the spatio-temporal structure of input sequences. Furthermore, these networks are surprisingly robust to shifts of the mean input strength and input spike rate, even when these shifts were not observed during training. As a consequence, high-performance networks can be obtained without any normalization techniques such as batch normalization or batch-normalization through time.

Training Spiking Deep Networks for Neuromorphic Hardware

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

IM-LIF: Improved Neuronal Dynamics with Attention Mechanism for Direct Training Deep Spiking Neural Network

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware.

Direct Training for Spiking Neural Networks: Faster, Larger, Better

Human-Level Control Through Directly-Trained Deep Spiking Q-Networks

High-performance deep spiking neural networks with 0.3 spikes per neuron

Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation

Learnable Surrogate Gradient for Direct Training Spiking Neural Networks

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation

Highly efficient neuromorphic learning system of spiking neural network with multi-compartment leaky integrate-and-fire neurons

The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

Improving classification accuracy of feedforward neural networks for spiking neuromorphic chips

An error-propagation spiking neural network compatible with neuromorphic processors

In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardware

A Digital Neuromorphic Hardware for Spiking Neural Network

Spiking Neurons with Neural Dynamics Implemented Using Stochastic Memristors