Abstract:Biological brains have the capability to adaptively coordinate relevant neuronal populations based on the task context to learn continuously changing tasks in real-world environments. However, existing spiking neural network-based continual learning algorithms treat each task equally, ignoring the guiding role of different task similarity associations for network learning, which limits knowledge utilization efficiency. Inspired by the context-dependent plasticity mechanism of the brain, we propose a Similarity-based Context Aware Spiking Neural Network (SCA-SNN) continual learning algorithm to efficiently accomplish task incremental learning and class incremental learning. Based on contextual similarity across tasks, the SCA-SNN model can adaptively reuse neurons from previous tasks that are beneficial for new tasks (the more similar, the more neurons are reused) and flexibly expand new neurons for the new task (the more similar, the fewer neurons are expanded). Selective reuse and discriminative expansion significantly improve the utilization of previous knowledge and reduce energy consumption. Extensive experimental results on CIFAR100, ImageNet generalized datasets, and FMNIST-MNIST, SVHN-CIFAR100 mixed datasets show that our SCA-SNN model achieves superior performance compared to both SNN-based and DNN-based continual learning algorithms. Additionally, our algorithm has the capability to adaptively select similar groups of neurons for related tasks, offering a promising approach to enhancing the biological interpretability of efficient continual learning.

Neural Similarity Learning

Deep Unsupervised Learning of Visual Similarities

Growing Deep Neural Network Considering with Similarity between Neurons

Visualizing Deep Similarity Networks

Inter-Class Angular Loss for Convolutional Neural Networks.

Bridging the Semantic Latent Space Between Brain and Machine: Similarity is All You Need

Image similarity using Deep CNN and Curriculum Learning

Similarity Learning with neural networks

Learning Non-Metric Visual Similarity for Image Retrieval

Learning similarity for semantic images classification

Graph-Based Similarity of Neural Network Representations.

Ercnn: Enhanced Recurrent Convolutional Neural Networks For Learning Sentence Similarity

Similarity-based context aware continual learning for spiking neural networks

Decoupling Semantic Similarity from Spatial Alignment for Neural Networks

Universal Correspondence Network

Similarity of Neural Network Representations Revisited

Learning visual similarity for product design with convolutional neural networks

Convolution in Convolution for Network in Network

To What Extent Do Different Neural Networks Learn the Same Representation: A Neuron Activation Subspace Match Approach

Convolutional Neural Networks Exploiting Attributes of Biological Neurons

Searching Similarity Measure for Binarized Neural Networks