Abstract:Humans can learn in a continuous manner. Old rarely utilized knowledge can be overwritten by new incoming information while important, frequently used knowledge is prevented from being erased. In artificial learning systems, lifelong learning so far has focused mainly on accumulating knowledge over tasks and overcoming catastrophic forgetting. In this paper, we argue that, given the limited model capacity and the unlimited new information to be learned, knowledge has to be preserved or erased selectively. Inspired by neuroplasticity, we propose a novel approach for lifelong learning, coined Memory Aware Synapses (MAS). It computes the importance of the parameters of a neural network in an unsupervised and online manner. Given a new sample which is fed to the network, MAS accumulates an importance measure for each parameter of the network, based on how sensitive the predicted output function is to a change in this parameter. When learning a new task, changes to important parameters can then be penalized, effectively preventing important knowledge related to previous tasks from being overwritten. Further, we show an interesting connection between a local version of our method and Hebb's rule,which is a model for the learning process in the brain. We test our method on a sequence of object recognition tasks and on the challenging problem of learning an embedding for predicting $<$subject, predicate, object$>$ triplets. We show state-of-the-art performance and, for the first time, the ability to adapt the importance of the parameters based on unlabeled data towards what the network needs (not) to forget, which may vary depending on test conditions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively and selectively retain or forget knowledge during the Lifelong Learning (LLL) process. Specifically, the author points out that under limited model capacity and in the face of infinite new information, traditional lifelong learning methods mainly focus on accumulating knowledge among tasks and overcoming catastrophic forgetting. However, when the model capacity reaches its limit, merely avoiding forgetting old knowledge is not enough; it is also necessary to be able to intelligently decide which knowledge can be forgotten to make room for learning new knowledge. For this purpose, inspired by neural plasticity, the author proposes a new method named "Memory Aware Synapses" (MAS) for lifelong learning. ### Core problems solved by the paper: 1. **Selective memory and forgetting**: How to intelligently decide which knowledge is important and needs to be retained and which knowledge can be forgotten under limited model capacity to adapt to the ever - changing learning environment. 2. **Unsupervised online learning**: How to use unlabeled data to dynamically adjust the importance of parameters so that the model can adapt according to the actual test conditions. 3. **Avoiding catastrophic forgetting**: How to prevent important knowledge of previous tasks from being overwritten when learning new tasks. ### Method overview: - **Parameter importance estimation**: MAS estimates the importance of each parameter by calculating the change in the sensitivity of network parameters to the output function. Specifically, for a given data point $x_k$, the gradient $g_{ij}(x_k)$ of the output function $F(x_k; \theta)$ with respect to the parameter $\theta_{ij}$ is used to measure the importance of the parameter. The importance weight $\Omega_{ij}$ is calculated by the following formula: \[ \Omega_{ij}=\frac{1}{N}\sum_{k = 1}^N\|g_{ij}(x_k)\| \] where $N$ is the total number of data points. - **New task learning**: When learning a new task, MAS adds a regularization term to penalize changes to important parameters, thereby preventing important knowledge from being overwritten. The specific form of the loss function is as follows: \[ L(\theta)=L_n(\theta)+\lambda\sum_{i,j}\Omega_{ij}(\theta_{ij}-\theta_{ij}^*)^2 \] where $L_n(\theta)$ is the loss function of the new task, $\lambda$ is the regularization parameter, and $\theta_{ij}^*$ is the optimized parameter value of the previous task. - **Connection with Hebbian learning**: The author also shows the connection between a local version of MAS and the Hebbian learning rule. Hebbian learning theory believes that "neurons that are activated together will be connected to each other", which is similar to the method of measuring parameter importance by the product of neuron activations in MAS. ### Experimental results: - **Object recognition tasks**: The author conducted experiments on multiple sequences of object recognition tasks, and the results show that MAS can effectively learn new tasks while maintaining the performance of old tasks and, in some cases, is superior to existing lifelong learning methods. - **Relationship prediction tasks**: The author also tested the performance of MAS on the task of learning to predict <subject, predicate, object> triples and also achieved good results. ### Conclusion: MAS dynamically adjusts the importance of parameters through unsupervised online learning, realizes the selective retention or forgetting of knowledge under limited model capacity, and thus effectively solves the key problems in lifelong learning.

Memory Aware Synapses: Learning what (not) to forget

Overcoming Long-Term Catastrophic Forgetting Through Adversarial Neural Pruning and Synaptic Consolidation

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

Attention-Based Structural-Plasticity

Meta-learning synaptic plasticity and memory addressing for continual familiarity detection

Representation Memorization for Fast Learning New Knowledge without Forgetting

Schematic Memory Persistence and Transience for Efficient and Robust Continual Learning

Anti-retroactive Interference for Lifelong Learning

Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting

Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting

Lifelong Learning With Cycle Memory Networks

Sparse Coding in a Dual Memory System for Lifelong Learning

GAN Memory with No Forgetting

Association: Remind Your GAN not to Forget

Overcoming catastrophic forgetting in neural networks

Incorporating neuro-inspired adaptability for continual learning in artificial intelligence

Overcome Anterograde Forgetting with Cycled Memory Networks

Fortuitous Forgetting in Connectionist Networks

Triple-Memory Networks: A Brain-Inspired Method for Continual Learning

ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks

Continual Memory: Can We Reason after Long-Term Memorization?