Abstract:To imitate the ability of keeping learning of human, continual learning which can learn from a never-ending data stream has attracted more interests recently. In all settings, the online class incremental learning (OCIL), where incoming samples from data stream can be used only once, is more challenging and can be encountered more frequently in real world. Actually, all continual learning models face a stability-plasticity dilemma, where the stability means the ability to preserve old knowledge while the plasticity denotes the ability to incorporate new knowledge. Although replay-based methods have shown exceptional promise, most of them concentrate on the strategy for updating and retrieving memory to keep stability at the expense of plasticity. To strike a preferable trade-off between stability and plasticity, we propose an Adaptive Focus Shifting algorithm (AFS), which dynamically adjusts focus to ambiguous samples and non-target logits in model learning. Through a deep analysis of the task-recency bias caused by class imbalance, we propose a revised focal loss to mainly keep stability. \Rt{By utilizing a new weight function, the revised focal loss will pay more attention to current ambiguous samples, which are the potentially valuable samples to make model progress quickly.} To promote plasticity, we introduce a virtual knowledge distillation. By designing a virtual teacher, it assigns more attention to non-target classes, which can surmount overconfidence and encourage model to focus on inter-class information. Extensive experiments on three popular datasets for OCIL have shown the effectiveness of AFS. The code will be available at \url{<a class="link-external link-https" href="https://github.com/czjghost/AFS" rel="external noopener nofollow">this https URL</a>}.

Task-Adaptive Saliency Guidance for Exemplar-free Class Incremental Learning

Efficient Statistical Sampling Adaptation for Exemplar-Free Class Incremental Learning

Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learning

TACLE: Task and Class-aware Exemplar-free Semi-supervised Class Incremental Learning

TaE: Task-aware Expandable Representation for Long Tail Class Incremental Learning

Class-Incremental Exemplar Compression for Class-Incremental Learning

Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning

Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt

CEAT: Continual Expansion and Absorption Transformer for Non-Exemplar Class-Incremental Learning

Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning

New Insights on Relieving Task-Recency Bias for Online Class Incremental Learning

Model Attention Expansion for Few-Shot Class-Incremental Learning

Fine-Grained Knowledge Selection and Restoration for Non-Exemplar Class Incremental Learning

Dynamic Integration of Task-Specific Adapters for Class Incremental Learning

Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning

DiffClass: Diffusion-Based Class Incremental Learning

eTag: Class-Incremental Learning with Embedding Distillation and Task-Oriented Generation

Towards Plastic and Stable Exemplar-Free Incremental Learning: A Dual-Learner Framework with Cumulative Parameter Averaging

Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks

Progressive Adapting and Pruning: Domain-Incremental Learning for Saliency Prediction