Emerging NeoHebbian Dynamics in Forward-Forward Learning: Implications for Neuromorphic Computing

Erik B. Terres-Escudero,Javier Del Ser,Pablo García-Bringas

2024-06-24

Abstract:Advances in neural computation have predominantly relied on the gradient backpropagation algorithm (BP). However, the recent shift towards non-stationary data modeling has highlighted the limitations of this heuristic, exposing that its adaptation capabilities are far from those seen in biological brains. Unlike BP, where weight updates are computed through a reverse error propagation path, Hebbian learning dynamics provide synaptic updates using only information within the layer itself. This has spurred interest in biologically plausible learning algorithms, hypothesized to overcome BP's shortcomings. In this context, Hinton recently introduced the Forward-Forward Algorithm (FFA), which employs local learning rules for each layer and has empirically proven its efficacy in multiple data modeling tasks. In this work we argue that when employing a squared Euclidean norm as a goodness function driving the local learning, the resulting FFA is equivalent to a neo-Hebbian Learning Rule. To verify this result, we compare the training behavior of FFA in analog networks with its Hebbian adaptation in spiking neural networks. Our experiments demonstrate that both versions of FFA produce similar accuracy and latent distributions. The findings herein reported provide empirical evidence linking biological learning rules with currently used training algorithms, thus paving the way towards extrapolating the positive outcomes from FFA to Hebbian learning rules. Simultaneously, our results imply that analog networks trained under FFA could be directly applied to neuromorphic computing, leading to reduced energy usage and increased computational speed.

Neural and Evolutionary Computing,Artificial Intelligence

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on the biological plausibility of neural network learning algorithms and their adaptability in non - stationary data modeling. Specifically:

1. **Learning algorithms with biological plausibility**: Traditional neural network training methods, such as the Backpropagation (BP) algorithm, perform well in many tasks, but their biological plausibility is questioned. The BP algorithm updates weights by back - propagating errors, which is inconsistent with the learning mechanism in the biological brain. The learning mechanism in the biological brain is more inclined to use local information for weight update, that is, the Hebbian learning rule. Therefore, this paper explores a new Forward - Forward Algorithm (FFA), which uses a local learning rule and aims to improve the biological plausibility of the algorithm.

2. **Non - stationary data modeling**: Modern data are often non - stationary, that is, the data distribution changes over time. The BP algorithm performs poorly in this case because it is difficult to adapt to the constantly changing data distribution. The FFA can better adapt to non - stationary data by using the local learning rule, thereby improving the adaptability and robustness of the model.

3. **The relationship between the Hebbian learning rule and FFA**: The paper further explores how the FFA is equivalent to a modulated Hebbian learning rule when the squared Euclidean norm is used as the goodness - of - fit function. This finding not only verifies the biological plausibility of the FFA but also provides a theoretical basis for applying the FFA to Spiking Neural Networks (SNN).

4. **The application potential in neuromorphic computing**: Based on the above research, the paper also discusses the application potential of the FFA in neuromorphic computing. Neuromorphic computing systems have the advantages of low power consumption and high computing speed, and the biological plausibility of the FFA makes it a promising learning algorithm in these systems.

In summary, the main objective of this paper is to verify the biological plausibility of the FFA by analyzing the relationship between the FFA and the Hebbian learning rule, and to explore its application potential in processing non - stationary data and neuromorphic computing.