Abstract:Few-shot class-incremental learning is crucial for developing scalable and adaptive intelligent systems, as it enables models to acquire new classes with minimal annotated data while safeguarding the previously accumulated knowledge. Nonetheless, existing methods deal with continuous data streams in a centralized manner, limiting their applicability in scenarios that prioritize data privacy and security. To this end, this paper introduces federated few-shot class-incremental learning, a decentralized machine learning paradigm tailored to progressively learn new classes from scarce data distributed across multiple clients. In this learning paradigm, clients locally update their models with new classes while preserving data privacy, and then transmit the model updates to a central server where they are aggregated globally. However, this paradigm faces several issues, such as difficulties in few-shot learning, catastrophic forgetting, and data heterogeneity. To address these challenges, we present a synthetic data-driven framework that leverages replay buffer data to maintain existing knowledge and facilitate the acquisition of new knowledge. Within this framework, a noise-aware generative replay module is developed to fine-tune local models with a balance of new and replay data, while generating synthetic data of new classes to further expand the replay buffer for future tasks. Furthermore, a class-specific weighted aggregation strategy is designed to tackle data heterogeneity by adaptively aggregating class-specific parameters based on local models performance on synthetic data. This enables effective global model optimization without direct access to client data. Comprehensive experiments across three widely-used datasets underscore the effectiveness and preeminence of the introduced framework.

Few-shot Class Incremental Learning Via Prompt Transfer and Knowledge Distillation

Class similarity weighted knowledge distillation for few shot incremental learning

Few-Shot Incremental Learning for Label-to-Image Translation

Uncertainty-Guided Semi-Supervised Few-Shot Class-Incremental Learning With Knowledge Distillation

Class-Incremental Few-Shot Event Detection

Progressive Network Grafting for Few-Shot Knowledge Distillation

Few-Shot Class-Incremental Learning Via Relation Knowledge Distillation

Few-Shot Class-Incremental Learning Via Class-Aware Bilateral Distillation

Class Incremental Learning with Deep Contrastive Learning and Attention Distillation

Few-Shot Class-Incremental Learning with Non-IID Decentralized Data

Class-Incremental Learning for Remote Sensing Images Based on Knowledge Distillation

Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning

Relation Knowledge Distillation Based on Prompt Learning for Generalized Few-Shot Intent Detection

Incremental Scene Classification Using Dual Knowledge Distillation and Classifier Discrepancy on Natural and Remote Sensing Images

M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning

Incremental Detection of Remote Sensing Objects with Feature Pyramid and Knowledge Distillation

Enhancing Few-Shot Learning in Lightweight Models via Dual-Faceted Knowledge Distillation

Adaptive knowledge transfer for class incremental learning

Self-supervised Knowledge Distillation for Few-shot Learning

Class incremental learning of remote sensing images based on class similarity distillation

Few-shot Class-Incremental Semantic Segmentation via Pseudo-Labeling and Knowledge Distillation