Abstract:Continual learning aims to emulate the human ability to continually accumulate knowledge over sequential tasks. The main challenge is to maintain performance on previously learned tasks after learning new tasks, i.e., to avoid catastrophic forgetting. We propose a Channel-wise Lightweight Reprogramming (CLR) approach that helps convolutional neural networks (CNNs) overcome catastrophic forgetting during continual learning. We show that a CNN model trained on an old task (or self-supervised proxy task) could be ``reprogrammed" to solve a new task by using our proposed lightweight (very cheap) reprogramming parameter. With the help of CLR, we have a better stability-plasticity trade-off to solve continual learning problems: To maintain stability and retain previous task ability, we use a common task-agnostic immutable part as the shared ``anchor" parameter set. We then add task-specific lightweight reprogramming parameters to reinterpret the outputs of the immutable parts, to enable plasticity and integrate new knowledge. To learn sequential tasks, we only train the lightweight reprogramming parameters to learn each new task. Reprogramming parameters are task-specific and exclusive to each task, which makes our method immune to catastrophic forgetting. To minimize the parameter requirement of reprogramming to learn new tasks, we make reprogramming lightweight by only adjusting essential kernels and learning channel-wise linear mappings from anchor parameters to task-specific domain knowledge. We show that, for general CNNs, the CLR parameter increase is less than 0.6\% for any new task. Our method outperforms 13 state-of-the-art continual learning baselines on a new challenging sequence of 53 image classification datasets. Code and data are available at <a class="link-external link-https" href="https://github.com/gyhandy/Channel-wise-Lightweight-Reprogramming" rel="external noopener nofollow">this https URL</a>

Learning After Learning: Positive Backward Transfer in Continual Learning

Progressive Learning without Forgetting

Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer

AFEC: Active Forgetting of Negative Transfer in Continual Learning

Learning with Recoverable Forgetting

Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion

Does Continual Learning Equally Forget All Parameters?

Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification

Bio-inspired, task-free continual learning through activity regularization

Slowing Down Forgetting in Continual Learning

Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning

Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

Prior-Free Continual Learning with Unlabeled Data in the Wild

Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learning

A Simple Lifelong Learning Approach

Continual Learning in the Presence of Spurious Correlation

AdaCL:Adaptive Continual Learning

CLR: Channel-wise Lightweight Reprogramming for Continual Learning

Forget but Recall: Incremental Latent Rectification in Continual Learning

Investigating the Impact of Weight Sharing Decisions on Knowledge Transfer in Continual Learning