Abstract:Deep neural networks often severely forget previously learned knowledge when learning new knowledge. Various continual learning (CL) methods have been proposed to handle such a catastrophic forgetting issue from different perspectives and achieved substantial improvements. In this paper, a novel two-branch continual learning framework is proposed to further enhance most existing CL methods. Specifically, the main branch can be any existing CL model and the newly introduced side branch is a lightweight convolutional network. The output of each main branch block is modulated by the output of the corresponding side branch block. Such a simple two-branch model can then be easily implemented and learned with the vanilla optimization setting without whistles and bells. Extensive experiments with various settings on multiple image datasets show that the proposed framework yields consistent improvements over state-of-the-art methods.

What problem does this paper attempt to address?

This paper aims to solve the problem that deep neural networks severely forget previously learned knowledge when learning new knowledge, namely catastrophic forgetting. Specifically, when a neural network is exposed to new classes or tasks during continuous learning, it often forgets the old knowledge it has already learned. This phenomenon is particularly evident in Class - Incremental Learning (CIL), because while the network is learning new classes, the recognition performance for old classes will decline significantly. To solve this problem, the author proposes a novel Generalizable Two - Branch (G2B) continuous learning framework to further enhance existing continuous learning methods. The main features of this framework are as follows: 1. **Framework Structure**: The G2B framework consists of two branches. The main branch can be any existing continuous learning model, while the side branch is a lightweight convolutional network. The output of each main branch block will be modulated by the output of the corresponding side branch block, thereby achieving a relatively sparse activation output. 2. **Modulation Mechanism**: By using the output of the side branch as a soft mask to modulate the output of the main branch, the output of the main branch becomes more sparse. This sparse output helps to reduce the forgetting of old knowledge during continuous learning. 3. **Compatibility**: The G2B framework is compatible with various continuous learning strategies based on Convolutional Neural Networks (CNN) or Vision Transformer (ViT), and the way of combining these strategies is very straightforward, and the optimization process is simple. 4. **Experimental Verification**: Through extensive experiments on multiple image datasets, it has been proven that the G2B framework can consistently improve the performance of existing methods, especially on datasets such as CIFAR - 100 and ImageNet, reaching a new state - of - the - art level. In conclusion, this paper effectively alleviates the problem of catastrophic forgetting in continuous learning by introducing a lightweight side - branch network and demonstrates its superior performance in various benchmark tests.

Generalizable Two-Branch Framework for Image Class-Incremental Learning

UniGrad-FS: Unified Gradient Projection with Flatter Sharpness for Continual Learning

Leveraging Old Knowledge to Continually Learn New Classes in Medical Images

A Unified and General Framework for Continual Learning

Self-Organizing Pathway Expansion for Non-Exemplar Class-Incremental Learning

CLUE: Consolidating Learned and Undergoing Experience in Domain-Incremental Classification

Class-Incremental Learning: Survey and Performance Evaluation on Image Classification

Forget but Recall: Incremental Latent Rectification in Continual Learning

Error-Driven Incremental Learning in Deep Convolutional Neural Network for Large-Scale Image Classification

Continual Recognition with Adaptive Memory Update.

Analysis of Continual Learning Techniques for Image Generative Models with Learned Class Information Management

Maintaining Discrimination and Fairness in Class Incremental Learning

Class-Incremental Learning: A Survey

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

CCSI: Continual Class-Specific Impression for Data-free Class Incremental Learning

A Classifier-Free Incremental Learning Framework for Scalable Medical Image Segmentation

Adaptive online continual multi-view learning

AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning

On the Exploration of Incremental Learning for Fine-grained Image Retrieval

Model Behavior Preserving for Class-Incremental Learning