Abstract:The subpopulation shifting challenge, known as some subpopulations of a category that are not seen during training, severely limits the classification performance of the state-of-the-art convolutional neural networks. Thus, to mitigate this practical issue, we explore incremental subpopulation learning (ISL) to adapt the original model via incrementally learning the unseen subpopulations without retaining the seen population data. However, striking a great balance between subpopulation learning and seen population forgetting is the main challenge in ISL but is not well studied by existing approaches. These incremental learners simply use a pre-defined and fixed hyperparameter to balance the learning objective and forgetting regularization, but their learning is usually biased towards either side in the long run. In this paper, we propose a novel two-stage learning scheme to explicitly disentangle the acquisition and forgetting for achieving a better balance between subpopulation learning and seen population forgetting: in the first “gain-acquisition” stage, we progressively learn a new classifier based on the margin-enforce loss, which enforces the hard samples and population to have a larger weight for classifier updating and avoid uniformly updating all the population; in the second “counter-forgetting” stage, we search for the proper combination of the new and old classifiers by optimizing a novel objective based on proxies of forgetting and acquisition. We benchmark the representative and state-of-the-art non-exemplar-based incremental learning methods on a large-scale subpopulation shifting dataset for the first time. Under almost all the challenging ISL protocols, we significantly outperform other methods by a large margin, demonstrating our superiority to alleviate the subpopulation shifting problem (Code is released in https://github.com/wuyujack/ISL).

ALRS: an Attention Loss Function Based on Row-Sparsity for Incremental Learning.

Learning with Recoverable Forgetting

Class Incremental Learning with Deep Contrastive Learning and Attention Distillation

Model Behavior Preserving for Class-Incremental Learning

Large Scale Incremental Learning

Maintaining Discrimination and Fairness in Class Incremental Learning

Class Incremental Learning with Less Forgetting Direction and Equilibrium Point

New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning

A Robust and Anti-Forgettiable Model for Class-Incremental Learning

Adaptive Rentention & Correction for Continual Learning

Incremental Land Cover Classification Via Label Strategy and Adaptive Weights

Balancing Between Forgetting and Acquisition in Incremental Subpopulation Learning.

Class Incremental Learning Based on Local Structure Constraints in Feature Space

SRIL: Selective Regularization for Class-Incremental Learning

Incremental Learning Based on Anchored Class Centers for SAR Automatic Target Recognition

Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning

EXACFS -- A CIL Method to mitigate Catastrophic Forgetting

Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning

Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild

SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models

Long-Tail Class Incremental Learning Via Independent Sub-prototype Construction