Abstract:To imitate the ability of keeping learning of human, continual learning which can learn from a never-ending data stream has attracted more interests recently. In all settings, the online class incremental learning (OCIL), where incoming samples from data stream can be used only once, is more challenging and can be encountered more frequently in real world. Actually, all continual learning models face a stability-plasticity dilemma, where the stability means the ability to preserve old knowledge while the plasticity denotes the ability to incorporate new knowledge. Although replay-based methods have shown exceptional promise, most of them concentrate on the strategy for updating and retrieving memory to keep stability at the expense of plasticity. To strike a preferable trade-off between stability and plasticity, we propose an Adaptive Focus Shifting algorithm (AFS), which dynamically adjusts focus to ambiguous samples and non-target logits in model learning. Through a deep analysis of the task-recency bias caused by class imbalance, we propose a revised focal loss to mainly keep stability. \Rt{By utilizing a new weight function, the revised focal loss will pay more attention to current ambiguous samples, which are the potentially valuable samples to make model progress quickly.} To promote plasticity, we introduce a virtual knowledge distillation. By designing a virtual teacher, it assigns more attention to non-target classes, which can surmount overconfidence and encourage model to focus on inter-class information. Extensive experiments on three popular datasets for OCIL have shown the effectiveness of AFS. The code will be available at \url{<a class="link-external link-https" href="https://github.com/czjghost/AFS" rel="external noopener nofollow">this https URL</a>}.

New Class Adaptation Via Instance Generation in One-Pass Class Incremental Learning

An Incremental One Class Learning Framework for Large Scale Data.

Learning to Classify With Incremental New Class

Bypassing Logits Bias in Online Class-Incremental Learning with a Generative Framework

One-Pass Learning with Incremental and Decremental Features

Adaptive knowledge transfer for class incremental learning

Self-Organizing Pathway Expansion for Non-Exemplar Class-Incremental Learning

Knowledge Adaptation Network for Few-Shot Class-Incremental Learning

New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning

Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective

Self-Paced Imbalance Rectification for Class Incremental Learning

Incremental Classifier Learning with Generative Adversarial Networks

A Class-Incremental Learning Method Based on One Class Support Vector Machine

OpenIncrement: A Unified Framework for Open Set Recognition and Deep Class-Incremental Learning

Prediction Error-based Classification for Class-Incremental Learning

Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion

Adaptive Discovering and Merging for Incremental Novel Class Discovery

Prototypes Sampling Mechanism for Class Incremental Learning

Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement

Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning

Few-shot Class-incremental Audio Classification Using Adaptively-refined Prototypes