Abstract:Federated Unlearning (FU) enables clients to selectively remove the influence of specific data from a trained federated learning model, addressing privacy concerns and regulatory requirements. However, existing FU methods often struggle to balance effective erasure with model utility preservation, especially for class-level unlearning in non-IID settings. We propose Federated Unlearning via Class-aware Representation Transformation (FUCRT), a novel method that achieves unlearning through class-aware representation transformation. FUCRT employs two key components: (1) a transformation class selection strategy to identify optimal forgetting directions, and (2) a transformation alignment technique using dual class-aware contrastive learning to ensure consistent transformations across clients. Extensive experiments on four datasets demonstrate FUCRT's superior performance in terms of erasure guarantee, model utility preservation, and efficiency. FUCRT achieves complete (100\%) erasure of unlearning classes while maintaining or improving performance on remaining classes, outperforming state-of-the-art baselines across both IID and Non-IID settings. Analysis of the representation space reveals FUCRT's ability to effectively merge unlearning class representations with the transformation class from remaining classes, closely mimicking the model retrained from scratch.

What problem does this paper attempt to address?

The paper attempts to address the problem of achieving data forgetting of specific target categories in Federated Learning (FL), known as Federated Unlearning (FU), i.e., selectively removing the influence of a specific dataset from a trained federated learning model without retraining the entire model. Specifically, the paper focuses on how to effectively balance the effectiveness of data forgetting and the retention of model performance in a Non-Independent and Identically Distributed (Non-IID) data setting, particularly in terms of category-level data forgetting. ### Background and Challenges 1. **Privacy Protection Needs**: With the introduction of regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), users have the right to request the deletion of their specific category data, which is an important requirement in federated learning. 2. **Limitations of Existing Methods**: - **Retraining the Model**: Although retraining the model is the most straightforward method, the computational and time costs are too high, making it impractical for real-world scenarios. - **Existing FU Methods**: Existing FU methods often struggle to find a balance between effective forgetting and maintaining model performance, especially in Non-IID data settings. ### Main Contributions of the Paper 1. **Exploring Federated Unlearning from the Perspective of Representation Space**: The paper is the first to explore the phenomenon of federated unlearning from the perspective of representation space and proposes two new observations. These observations indicate that the forgotten data and the remaining data are not completely opposed but can be transformed within the representation space. 2. **Proposing the FUCRT Method**: The paper proposes a new method—Federated Unlearning via Class-aware Representation Transformation (FUCRT). FUCRT is implemented through the following two key components: - **Transformation Category Selection Strategy**: Selecting the optimal forgetting direction. - **Transformation Alignment Technique**: Using dual class-aware contrastive learning to ensure consistent transformation across clients. 3. **Experimental Validation**: Extensive experimental results validate the superiority of FUCRT in terms of forgetting assurance, model performance retention, and efficiency. FUCRT can completely (100%) forget the target category while maintaining or improving the performance of the remaining categories, outperforming existing baseline methods in both IID and Non-IID settings. ### Method Overview 1. **Transformation Category Selection**: - **Generating a Global Transformation Category Set**: Using the local data distribution of each client to generate a local transformation category set, then aggregating them on the server to generate a global transformation category set. - **Selecting Transformation Categories for Each Forgotten Sample**: Based on the global transformation category set and the current knowledge of the model, selecting the most appropriate transformation category for each forgotten sample. 2. **Transformation Alignment**: - **Local and Global Class-aware Contrastive Learning**: Optimizing the local class-aware representation space through local class-aware contrastive learning, while aligning it with the global class-aware representation space through global class-aware contrastive learning, ensuring the reliability and consistency of the representation transformation of the forgotten data within the representation space. ### Conclusion The FUCRT method achieves effective category-level data forgetting in federated learning while maintaining model performance. By exploring the forgetting process from the perspective of representation space, the method proposes new transformation category selection and alignment techniques, addressing the limitations of existing methods in Non-IID data settings.

Forgetting Through Transforming: Enabling Federated Unlearning via Class-Aware Representation Transformation

Enable the Right to be Forgotten with Federated Client Unlearning in Medical Imaging

Streamlined Federated Unlearning: Unite as One to Be Highly Efficient

Forgettable Federated Linear Learning with Certified Data Unlearning

A Survey on Federated Unlearning: Challenges, Methods, and Future Directions

Unlearning during Learning: An Efficient Federated Machine Unlearning Method

Upcycling Noise for Federated Unlearning

Federated Unlearning via Active Forgetting

Communication Efficient and Provable Federated Unlearning

Towards Federated Domain Unlearning: Verification Methodologies and Challenges

Subspace Based Federated Unlearning

Privacy-Preserving Federated Unlearning with Certified Client Removal

Federated Unlearning Via Class-Discriminative Pruning

The Right to Be Forgotten in Federated Learning: an Efficient Realization with Rapid Retraining

Guaranteeing Data Privacy in Federated Unlearning with Dynamic User Participation

Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics

FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method

TARGET: Federated Class-Continual Learning Via Exemplar-Free Distillation

FedFRR: Federated Forgetting-Resistant Representation Learning

Adaptive Clipping and Distillation Enabled Federated Unlearning