Abstract:Recently, the use of bio-inspired learning techniques such as Hebbian learning and its closely-related Spike-Timing-Dependent Plasticity (STDP) variant have drawn significant attention for the design of compute-efficient AI systems that can continuously learn on-line at the edge. A key differentiating factor regarding this emerging class of neuromorphic continual learning system lies in the fact that learning must be carried using a data stream received in its natural order, as opposed to conventional gradient-based offline training, where a static training dataset is assumed available a priori and randomly shuffled to make the training set independent and identically distributed (i.i.d). In contrast, the emerging class of neuromorphic continual learning systems covered in this survey must learn to integrate new information on the fly in a non-i.i.d manner, which makes these systems subject to catastrophic forgetting. In order to build the next generation of neuromorphic AI systems that can continuously learn at the edge, a growing number of research groups are studying the use of Sparse and Predictive Coding-based Hebbian neural network architectures and the related Spiking Neural Networks (SNNs) equipped with STDP learning. However, since this research field is still emerging, there is a need for providing a holistic view of the different approaches proposed in the literature so far. To this end, this survey covers a number of recent works in the field of neuromorphic continual learning based on state-of-the-art Sparse and Predictive Coding technology; provides background theory to help interested researchers quickly learn the key concepts; and discusses important future research questions in light of the different works covered in this paper. It is hoped that this survey will contribute towards future research in the field of neuromorphic continual learning.

Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Similarity-based context aware continual learning for spiking neural networks

Enhancing Efficient Continual Learning with Dynamic Structure Development of Spiking Neural Networks

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

Adaptive Smoothing Gradient Learning for Spiking Neural Networks.

Spiking Deep Residual Networks.

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness

ALADE-SNN: Adaptive Logit Alignment in Dynamically Expandable Spiking Neural Networks for Class Incremental Learning

Continual Learning with Neuromorphic Computing: Theories, Methods, and Applications

Continual Learning with Hebbian Plasticity in Sparse and Predictive Coding Networks: A Survey and Perspective

Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling

High-Performance Temporal Reversible Spiking Neural Networks with $O(L)$ Training Memory and $O(1)$ Inference Cost

Autonomous Cognition Development with Lifelong Learning: A Self-Organizing and Reflecting Cognitive Network

Backpropagation with Sparsity Regularization for Spiking Neural Network Learning.

Memory-Efficient Reversible Spiking Neural Networks

Incorporating structural plasticity into self-organization recurrent networks for sequence learning