Event-based backpropagation on the neuromorphic platform SpiNNaker2

Béna Gabriel,Wunderlich Timo,Akl Mahmoud,Vogginger Bernhard,Mayr Christian,Andres Gonzales Hector

2024-12-20

Abstract:Neuromorphic computing aims to replicate the brain's capabilities for energy efficient and parallel information processing, promising a solution to the increasing demand for faster and more efficient computational systems. Efficient training of neural networks on neuromorphic hardware requires the development of training algorithms that retain the sparsity of spike-based communication during training. Here, we report on the first implementation of event-based backpropagation on the SpiNNaker2 neuromorphic hardware platform. We use EventProp, an algorithm for event-based backpropagation in spiking neural networks (SNNs), to compute exact gradients using sparse communication of error signals between neurons. Our implementation computes multi-layer networks of leaky integrate-and-fire neurons using discretized versions of the differential equations and their adjoints, and uses event packets to transmit spikes and error signals between network layers. We demonstrate a proof-of-concept of batch-parallelized, on-chip training of SNNs using the Yin Yang dataset, and provide an off-chip implementation for efficient prototyping, hyper-parameter search, and hybrid training methods.

Neural and Evolutionary Computing,Hardware Architecture,Emerging Technologies

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: Implement the event - based backpropagation algorithm on the neuromorphic computing platform SpiNNaker2 to effectively train spiking neural networks (SNNs). Specifically, the researchers aim to develop a training algorithm that can maintain the sparsity of spiking communication during the training process, thereby improving the efficiency and energy utilization of neural network training. By using the EventProp algorithm, they achieve the calculation of accurate gradients and use sparse error - signal communication to update the weights between neurons.

The formula in Markdown format is as follows:

- The time - to - first - spike loss in the loss function is defined as:
\[
L = -\log \left( \frac{\exp(-t_{s_l} \Delta t / \tau_0)}{\sum_{k = 1}^{3}\exp(-t_{s_k} \Delta t / \tau_0)} \right)-\alpha \left[\exp\left(\frac{t_{s_l} \Delta t}{\tau_1}\right)- 1\right]
\]

where \( t_{s_k} \) is the time step of the first spike of the \( k \) - th neuron, \( l \) is the index of the neuron that should fire first, and \( \tau_0, \tau_1 \) and \( \alpha \) are the hyper - parameters of the loss function.

Through this method, the researchers hope to achieve more efficient and energy - saving deep - learning model training on neuromorphic hardware platforms, especially for tasks that need to process time - series data. In addition, this study also explores how to apply this event - based backpropagation method to online - learning scenarios, enabling neural networks to adapt to changing data streams in real - time.