Abstract:As Web technology continues to develop, it has become increasingly common to use data stored on different clients. At the same time, federated learning has received widespread attention due to its ability to protect data privacy when let models learn from data which is distributed across various clients. However, most existing works assume that the client's data are fixed. In real-world scenarios, such an assumption is most likely not true as data may be continuously generated and new classes may also appear. To this end, we focus on the practical and challenging federated class-incremental learning (FCIL) problem. For FCIL, the local and global models may suffer from catastrophic forgetting on old classes caused by the arrival of new classes and the data distributions of clients are non-independent and identically distributed (non-iid). In this paper, we propose a novel method called Federated Class-Incremental Learning with PrompTing (FCILPT). Given the privacy and limited memory, FCILPT does not use a rehearsal-based buffer to keep exemplars of old data. We choose to use prompts to ease the catastrophic forgetting of the old classes. Specifically, we encode the task-relevant and task-irrelevant knowledge into prompts, preserving the old and new knowledge of the local clients and solving the problem of catastrophic forgetting. We first sort the task information in the prompt pool in the local clients to align the task information on different clients before global aggregation. It ensures that the same task's knowledge are fully integrated, solving the problem of non-iid caused by the lack of classes among different clients in the same incremental task. Experiments on CIFAR-100, Mini-ImageNet, and Tiny-ImageNet demonstrate that FCILPT achieves significant accuracy improvements over the state-of-the-art methods.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address several key challenges in Federated Class-Incremental Learning (FCIL): 1. **Catastrophic Forgetting**: When new classes appear, both local and global models may forget old classes. This is due to the continuous generation of new data and uneven distribution. 2. **Non-iid**: The data distribution across different clients is inconsistent, which can lead to a decline in the performance of the aggregated model. 3. **Privacy Protection**: In federated learning, data privacy needs to be protected, and memory buffers cannot be used to store examples of old data. ### Specific Problem Description - **Limitations of Existing Methods**: Most existing federated learning methods assume that the data of each local client is fixed. However, in real-world scenarios, local clients may continuously generate or receive new data, and this new data may contain new classes. In such cases, existing federated learning methods need to retrain the model from scratch, which is not only costly but also unsustainable. - **Challenges of Class-Incremental Learning**: Class-Incremental Learning (CIL) can avoid the high cost of retraining through incremental learning, but most existing CIL methods do not consider the federated learning setup, facing issues such as memory limitations, privacy security, and poor performance. - **Non-iid Problem**: The inconsistent data distribution among different clients further exacerbates the problem of catastrophic forgetting. ### Solution To address the above issues, the authors propose a new method—**Federated Class-Incremental Learning with PrompTing (FCILPT)**. Specifically: 1. **Prompt Encoding**: FCILPT uses prompts to alleviate catastrophic forgetting. Prompts can encode task-related and task-unrelated knowledge, thereby retaining old and new knowledge without relying on memory buffers. 2. **Instance-Level Prompt Query Mechanism**: By introducing a learnable key-value mapping mechanism, FCILPT can accurately select the appropriate prompt for each instance without knowing the current task identity. 3. **Task-Unrelated Prompts**: To capture potential general knowledge, FCILPT introduces task-unrelated prompts that can aggregate knowledge from all arrived tasks. 4. **Prompt Pool Sorting and Alignment**: Before aggregation on the server side, the prompt pools of local clients are sorted and aligned to ensure that the task information of different clients is fully integrated, addressing the non-iid problem caused by the lack of classes. ### Experimental Results Experiments were conducted on three benchmark datasets: CIFAR-100, Mini-ImageNet, and Tiny-ImageNet. The results show that FCILPT significantly outperforms existing state-of-the-art methods. ### Summary This paper proposes a new method, FCILPT, to address the issues of catastrophic forgetting and non-iid in federated class-incremental learning. By using prompt encoding, an instance-level prompt query mechanism, and task-unrelated prompts, FCILPT effectively alleviates these problems without relying on memory buffers and achieves significant performance improvements on multiple benchmark datasets.

Federated Class-Incremental Learning with Prompting