Abstract:Federated Class-Incremental Learning (FCIL) focuses on continually transferring the previous knowledge to learn new classes in dynamic Federated Learning (FL). However, existing methods do not consider the trustworthiness of FCIL, i.e., improving continual utility, privacy, and efficiency simultaneously, which is greatly influenced by catastrophic forgetting and data heterogeneity among clients. To address this issue, we propose FedProK (Federated Prototypical Feature Knowledge Transfer), leveraging prototypical feature as a novel representation of knowledge to perform spatial-temporal knowledge transfer. Specifically, FedProK consists of two components: (1) feature translation procedure on the client side by temporal knowledge transfer from the learned classes and (2) prototypical knowledge fusion on the server side by spatial knowledge transfer among clients. Extensive experiments conducted in both synchronous and asynchronous settings demonstrate that our FedProK outperforms the other state-of-the-art methods in three perspectives of trustworthiness, validating its effectiveness in selectively transferring spatial-temporal knowledge.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve trustworthy Federated Class - Incremental Learning (FCIL) in the Federated Learning (FL) environment. Specifically, the paper focuses on how to simultaneously improve continuous utility, privacy protection, and efficiency in dynamic and heterogeneous Federated Learning scenarios, especially when facing two key challenges: catastrophic forgetting and data heterogeneity. ### Background and Problem Description **Background**: - **Federated Learning (FL)**: A distributed machine - learning method that allows multiple clients to collaborate in training a model without sharing data. - **Class - Incremental Learning (CIL)**: A continuous - learning method aimed at enabling the model to learn new knowledge without forgetting old knowledge when new classes appear. **Problem**: - **Catastrophic Forgetting**: When the model is learning a new task, it is prone to forget the knowledge of previously learned tasks. - **Data Heterogeneity**: The data distributions of different clients vary greatly, resulting in a decline in the performance of the global model and unstable convergence. ### Limitations of Existing Methods - **Storing or Generating Samples**: Existing methods usually relieve catastrophic forgetting by storing or generating samples of old classes and replaying these samples when training new tasks. However, this method is limited by the size of the sample set and may exceed the memory capacity of edge devices. - **Communication Costs and Privacy Issues**: Some methods generate pseudo - samples on the server and distribute them to clients, which will increase communication costs and introduce potential security risks. - **Balancing Stability and Plasticity**: Existing FCIL methods fail to well balance the model's retention of old knowledge (stability) and its adaptation to new knowledge (plasticity). ### The Solution Proposed in the Paper To address the above problems, the paper proposes **FedProK (Federated Prototypical Feature Knowledge Transfer)**, a trustworthy Federated Class - Incremental Learning framework. FedProK achieves spatio - temporal knowledge transfer through the following two components: 1. **Feature Translation**: - **Temporal Knowledge Transfer**: On the client side, the feature translation mechanism is used to transfer the features of old classes to new classes to alleviate catastrophic forgetting. - **Feature Translation Process**: \[ \hat{f}_t(x_p)=f(x_n)+\mu_t^{k,p}-\mu_t^{k,n} \] where \( f(x_n) \) is the real - sample feature of the new class \( n \), and \( \mu_t^{k,p} \) and \( \mu_t^{k,n} \) are the prototypes of the old class \( p \) and the new class \( n \), respectively. 2. **Prototypical Knowledge Fusion**: - **Spatial Knowledge Transfer**: On the server side, the prototypical knowledge fusion mechanism is used to transfer knowledge among clients to solve the problem of data heterogeneity. - **Fusion Process**: \[ \mu_t^c = \begin{cases} \frac{\sum_{k = 1}^K|D_k^c|\mu_r^{k,c}}{|D_c|}, & \text{if } c\notin\cup_{i = 1}^{t - 1}C_i \\ \beta\frac{\sum_{k = 1}^K|D_k^c|\mu_r^{k,c}}{|D_c|}+(1 - \beta)\mu_{t - 1}^c \end{cases} \]

FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfer

FedSKF: Selective Knowledge Fusion via Optimal Transport in Federated Class Incremental Learning

Federated Class-Incremental Learning with New-Class Augmented Self-Distillation

Edge-cloud Collaborative Learning with Federated and Centralized Features

TARGET: Federated Class-Continual Learning Via Exemplar-Free Distillation

Federated Class-Incremental Learning with Prompting

pFedKT: Personalized federated learning with dual knowledge transfer

FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework

A Prototype-Based Knowledge Distillation Framework for Heterogeneous Federated Learning

Overcoming Catastrophic Forgetting in Federated Class-Incremental Learning via Federated Global Twin Generator

Data-Free Federated Class Incremental Learning with Diffusion-Based Generative Memory

Knowledge Lock: Overcoming Catastrophic Forgetting in Federated Learning

Personalized Federated Domain-Incremental Learning based on Adaptive Knowledge Matching

FedINC: an Exemplar-Free Continual Federated Learning Framework with Small Labeled Data

PIP: Prototypes-Injected Prompt for Federated Class Incremental Learning

A Hierarchical Knowledge Transfer Framework for Heterogeneous Federated Learning.

Dynamic Heterogeneous Federated Learning with Multi-Level Prototypes

An Enhanced Federated Prototype Learning Method under Domain Shift

Federated Class-Incremental Learning with Hierarchical Generative Prototypes

Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning

Improving Global Generalization and Local Personalization for Federated Learning