Abstract:The subpopulation shifting challenge, known as some subpopulations of a category that are not seen during training, severely limits the classification performance of the state-of-the-art convolutional neural networks. Thus, to mitigate this practical issue, we explore incremental subpopulation learning (ISL) to adapt the original model via incrementally learning the unseen subpopulations without retaining the seen population data. However, striking a great balance between subpopulation learning and seen population forgetting is the main challenge in ISL but is not well studied by existing approaches. These incremental learners simply use a pre-defined and fixed hyperparameter to balance the learning objective and forgetting regularization, but their learning is usually biased towards either side in the long run. In this paper, we propose a novel two-stage learning scheme to explicitly disentangle the acquisition and forgetting for achieving a better balance between subpopulation learning and seen population forgetting: in the first “gain-acquisition” stage, we progressively learn a new classifier based on the margin-enforce loss, which enforces the hard samples and population to have a larger weight for classifier updating and avoid uniformly updating all the population; in the second “counter-forgetting” stage, we search for the proper combination of the new and old classifiers by optimizing a novel objective based on proxies of forgetting and acquisition. We benchmark the representative and state-of-the-art non-exemplar-based incremental learning methods on a large-scale subpopulation shifting dataset for the first time. Under almost all the challenging ISL protocols, we significantly outperform other methods by a large margin, demonstrating our superiority to alleviate the subpopulation shifting problem (Code is released in https://github.com/wuyujack/ISL).

A self-organizing incremental neural network for imbalance learning

Learning a Unified Classifier Incrementally via Rebalancing

Self-Paced Imbalance Rectification for Class Incremental Learning

Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective

A Self-Organizing Incremental Neural Network Based on Local Distribution Learning

Multi-Granularity Regularized Re-Balancing for Class Incremental Learning

Self-Organizing Incremental Neural Network And Its Application

A neural network learning algorithm for highly imbalanced data classification

Large Scale Incremental Learning

Discrimination Correction and Balance for Class-Incremental Learning

Balancing Between Forgetting and Acquisition in Incremental Subpopulation Learning.

Dual Balanced Class-Incremental Learning With im-Softmax and Angular Rectification

More Task-Balanced Class-Incremental Learning

Gradient Reweighting: Towards Imbalanced Class-Incremental Learning

An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network

Balanced softmax cross-entropy for incremental learning with and without memory

Class-incremental learning with Balanced Embedding Discrimination Maximization

Tackle Balancing Constraint for Incremental Semi-Supervised Support Vector Learning

Oversampling Algorithm based on Reinforcement Learning in Imbalanced Problems

Clustering-based incremental learning for imbalanced data classification

Handling Inter-class and Intra-class Imbalance in Class-imbalanced Learning