Abstract:Efficient implementations of spiking neural networks on neuromorphic hardware promise orders of magnitude less power consumption than their non-spiking counterparts. The standard neuron model for spike-based computation on such neuromorphic systems has long been the leaky integrate-and-fire (LIF) neuron. As a promising advancement, a computationally light augmentation of the LIF neuron model with an adaptation mechanism experienced a recent upswing in popularity, caused by demonstrations of its superior performance on spatio-temporal processing tasks. The root of the superiority of these so-called adaptive LIF neurons however, is not well understood. In this article, we thoroughly analyze the dynamical, computational, and learning properties of adaptive LIF neurons and networks thereof. We find that the frequently observed stability problems during training of such networks can be overcome by applying an alternative discretization method that results in provably better stability properties than the commonly used Euler-Forward method. With this discretization, we achieved a new state-of-the-art performance on common event-based benchmark datasets. We also show that the superiority of networks of adaptive LIF neurons extends to the prediction and generation of complex time series. Our further analysis of the computational properties of networks of adaptive LIF neurons shows that they are particularly well suited to exploit the spatio-temporal structure of input sequences. Furthermore, these networks are surprisingly robust to shifts of the mean input strength and input spike rate, even when these shifts were not observed during training. As a consequence, high-performance networks can be obtained without any normalization techniques such as batch normalization or batch-normalization through time.

Active Dendrites Enable Efficient Continual Learning in Time-To-First-Spike Neural Networks

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

A Sparsity-Adapted Hardware Implementation of SNN for Cortical Spike Trains Decoding

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

NADOL: Neuromorphic Architecture for Spike-Driven Online Learning by Dendrites

An Edge Neuromorphic Hardware With Fast On-Chip Error-Triggered Learning on Compressive Sensed Spikes

MT-Spike: A Multilayer Time-based Spiking Neuromorphic Architecture with Temporal Error Backpropagation

Supervised Learning with First-to-Spike Decoding in Multilayer Spiking Neural Networks

Enabling Efficient On-Edge Spiking Neural Network Acceleration with Highly Flexible FPGA Architectures

Efficient Spike-Driven Learning With Dendritic Event-Based Processing

Improving classification accuracy of feedforward neural networks for spiking neuromorphic chips

PT-Spike: A Precise-Time-Dependent Single Spike Neuromorphic Architecture with Efficient Supervised Learning

Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation

TripleBrain: An Edge Neuromorphic Architecture for High-accuracy Single-layer Spiking Neural Network with On-chip Self-organizing and Reinforcement Learning

Continuous Learning in a Single-Incremental-Task Scenario with Spike Features

Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks

A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps

You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Stochastic dendrites enable online learning in mixed-signal neuromorphic processing systems