How to Leverage Predictive Uncertainty Estimates for Reducing Catastrophic Forgetting in Online Continual Learning

Giuseppe Serra,Ben Werner,Florian Buettner

2024-10-10

Abstract:Many real-world applications require machine-learning models to be able to deal with non-stationary data distributions and thus learn autonomously over an extended period of time, often in an online setting. One of the main challenges in this scenario is the so-called catastrophic forgetting (CF) for which the learning model tends to focus on the most recent tasks while experiencing predictive degradation on older ones. In the online setting, the most effective solutions employ a fixed-size memory buffer to store old samples used for replay when training on new tasks. Many approaches have been presented to tackle this problem. However, it is not clear how predictive uncertainty information for memory management can be leveraged in the most effective manner and conflicting strategies are proposed to populate the memory. Are the easiest-to-forget or the easiest-to-remember samples more effective in combating CF? Starting from the intuition that predictive uncertainty provides an idea of the samples' location in the decision space, this work presents an in-depth analysis of different uncertainty estimates and strategies for populating the memory. The investigation provides a better understanding of the characteristics data points should have for alleviating CF. Then, we propose an alternative method for estimating predictive uncertainty via the generalised variance induced by the negative log-likelihood. Finally, we demonstrate that the use of predictive uncertainty measures helps in reducing CF in different settings.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use predictive uncertainty estimation to reduce catastrophic forgetting (CF) in online continual learning (online - CL). Specifically, the paper explores how machine - learning models can learn autonomously over a long period of time when dealing with non - stationary data distributions, and not forget old tasks while learning new ones. The paper pays particular attention to how to optimize the memory management strategy through predictive uncertainty information, thereby effectively alleviating the phenomenon of catastrophic forgetting.

The core problems of the paper can be summarized as follows:
1. How to use predictive uncertainty estimation to reduce catastrophic forgetting in online continual learning?
2. What are the effects of different uncertainty estimation methods on alleviating catastrophic forgetting?
3. Which is more effective in alleviating catastrophic forgetting, the most representative samples (the easiest to remember) or the marginal samples (the easiest to forget)?

Through these problems, the paper aims to provide a new method to optimize the memory management strategy in online continual learning, thereby improving the learning efficiency and performance of the model.