Abstract:Class Incremental Semantic Segmentation~(CISS), within Incremental Learning for semantic segmentation, targets segmenting new categories while reducing the catastrophic forgetting on the old categories.Besides, background shifting, where the background category changes constantly in each step, is a special challenge for CISS. Current methods with a shared background classifier struggle to keep up with these changes, leading to decreased stability in background predictions and reduced accuracy of segmentation. For this special challenge, we designed a novel background adaptation mechanism, which explicitly models the background residual rather than the background itself in each step, and aggregates these residuals to represent the evolving background. Therefore, the background adaptation mechanism ensures the stability of previous background classifiers, while enabling the model to concentrate on the easy-learned residuals from the additional channel, which enhances background discernment for better prediction of novel categories. To precisely optimize the background adaptation mechanism, we propose Pseudo Background Binary Cross-Entropy loss and Background Adaptation losses, which amplify the adaptation effect. Group Knowledge Distillation and Background Feature Distillation strategies are designed to prevent forgetting old categories. Our approach, evaluated across various incremental scenarios on Pascal VOC 2012 and ADE20K datasets, outperforms prior exemplar-free state-of-the-art methods with mIoU of 3.0% in VOC 10-1 and 2.0% in ADE 100-5, notably enhancing the accuracy of new classes while mitigating catastrophic forgetting. Code is available in <a class="link-external link-https" href="https://andyzaq.github.io/barmsite/" rel="external noopener nofollow">this https URL</a>.

Adaptive Prototype Replay for Class Incremental Semantic Segmentation

Saving 100x Storage: Prototype Replay for Reconstructing Training Sample Distribution in Class-Incremental Semantic Segmentation

Background Adaptation with Residual Modeling for Exemplar-Free Class-Incremental Semantic Segmentation

Inherit With Distillation and Evolve With Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory

Task Consistent Prototype Learning for Incremental Few-shot Semantic Segmentation

DiffusePast: Diffusion-based Generative Replay for Class Incremental Semantic Segmentation

SAM-Adapter: Adapting Segment Anything in Underperformed Scenes

Condensed Prototype Replay for Class Incremental Learning

Prototypes Sampling Mechanism for Class Incremental Learning

Towards Realistic Incremental Scenario in Class Incremental Semantic Segmentation

Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning

Adaptive Prototype Learning and Allocation for Few-Shot Segmentation

Cs2K: Class-specific and Class-shared Knowledge Guidance for Incremental Semantic Segmentation

CIT: Rethinking Class-incremental Semantic Segmentation with a Class Independent Transformation

Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting

Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation.

Prototype Representation Expansion in Incremental Learning

Continual Semantic Segmentation with Automatic Memory Sample Selection

Continual Recognition with Adaptive Memory Update.

Class-Prompting Transformer for Incremental Semantic Segmentation

MixER: Mixup-Based Experience Replay for Online Class-Incremental Learning