Abstract:Efficient implementations of spiking neural networks on neuromorphic hardware promise orders of magnitude less power consumption than their non-spiking counterparts. The standard neuron model for spike-based computation on such neuromorphic systems has long been the leaky integrate-and-fire (LIF) neuron. As a promising advancement, a computationally light augmentation of the LIF neuron model with an adaptation mechanism experienced a recent upswing in popularity, caused by demonstrations of its superior performance on spatio-temporal processing tasks. The root of the superiority of these so-called adaptive LIF neurons however, is not well understood. In this article, we thoroughly analyze the dynamical, computational, and learning properties of adaptive LIF neurons and networks thereof. We find that the frequently observed stability problems during training of such networks can be overcome by applying an alternative discretization method that results in provably better stability properties than the commonly used Euler-Forward method. With this discretization, we achieved a new state-of-the-art performance on common event-based benchmark datasets. We also show that the superiority of networks of adaptive LIF neurons extends to the prediction and generation of complex time series. Our further analysis of the computational properties of networks of adaptive LIF neurons shows that they are particularly well suited to exploit the spatio-temporal structure of input sequences. Furthermore, these networks are surprisingly robust to shifts of the mean input strength and input spike rate, even when these shifts were not observed during training. As a consequence, high-performance networks can be obtained without any normalization techniques such as batch normalization or batch-normalization through time.

Fast learning without synaptic plasticity in spiking neural networks

A solution to the learning dilemma for recurrent networks of spiking neurons

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Temporal Coding in Recurrent Spiking Neural Networks with Synaptic Delay-Weight Plasticity

Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rules

Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation

The brain as an efficient and robust adaptive learner

A framework for studying synaptic plasticity with neural spike train data

Extending dynamic memory of spiking neuron networks

Optimality of short-term synaptic plasticity in modelling certain dynamic environments

Memory-Dependent Computation and Learning in Spiking Neural Networks Through Hebbian Plasticity

Fast and slow synaptic plasticity enables concurrent control and learning

Nonlinear slow-timescale mechanisms in synaptic plasticity

Brain-Inspired Hardware for Artificial Intelligence: Accelerated Learning in a Physical-Model Spiking Neural Network

Computing of temporal information in spiking neural networks with ReRAM synapses

Memory-efficient neurons and synapses for spike-timing-dependent-plasticity in large-scale spiking networks

Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks

Fast and robust learning in Spiking Feed-forward Neural Networks based on Intrinsic Plasticity mechanism.

Learning fast changing slow in spiking neural networks

Learning recurrent dynamics in spiking networks