Abstract:The advent of Federated Learning (FL) highlights the practical necessity for the 'right to be forgotten' for all clients, allowing them to request data deletion from the machine learning model's service provider. This necessity has spurred a growing demand for Federated Unlearning (FU). Feature unlearning has gained considerable attention due to its applications in unlearning sensitive features, backdoor features, and bias features. Existing methods employ the influence function to achieve feature unlearning, which is impractical for FL as it necessitates the participation of other clients in the unlearning process. Furthermore, current research lacks an evaluation of the effectiveness of feature unlearning. To address these limitations, we define feature sensitivity in the evaluation of feature unlearning according to Lipschitz continuity. This metric characterizes the rate of change or sensitivity of the model output to perturbations in the input feature. We then propose an effective federated feature unlearning framework called Ferrari, which minimizes feature sensitivity. Extensive experimental results and theoretical analysis demonstrate the effectiveness of Ferrari across various feature unlearning scenarios, including sensitive, backdoor, and biased features.

What problem does this paper attempt to address?

The paper attempts to address the problem of achieving feature unlearning in a Federated Learning (FL) environment. Specifically, the paper focuses on how to effectively remove specific features from a federated learning model without retraining the entire model, while maintaining the model's performance on other data. ### Background and Challenges 1. **Privacy Protection Requirements**: - Participants in federated learning have the right to request the removal of their data's influence on the model, as explicitly stated by regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), which include the "right to be forgotten." - Existing federated unlearning methods mainly focus on client-level, category-level, or sample-level unlearning, but lack research on feature-level unlearning. 2. **Challenges of Feature Unlearning**: - **Evaluation Difficulty**: Evaluating the effectiveness of feature unlearning typically requires comparing the unlearned model with a model retrained on data that does not include the specific feature, but constructing such a dataset is very challenging. - **Dependence on Other Clients**: Existing feature unlearning methods usually require access to all datasets, which is impractical in federated learning because other clients may be unwilling or unable to share data or computational resources. ### Main Contributions of the Paper 1. **Definition of Feature Sensitivity**: - Introduced the concept of Feature Sensitivity, which measures the sensitivity of the model output to perturbations in the input features through Lipschitz continuity. - The smaller the feature sensitivity, the less the model remembers the specific feature. 2. **Proposed Ferrari Framework**: - Ferrari is an efficient federated feature unlearning framework that achieves feature unlearning by minimizing feature sensitivity. - Ferrari only requires the local dataset of the target client, without the need for participation from other clients, thus improving practicality and efficiency. 3. **Theoretical Analysis and Experimental Validation**: - Demonstrated that Ferrari can achieve lower model utility loss during the feature unlearning process, outperforming precise feature unlearning. - Extensive experimental results validated Ferrari's effectiveness in different scenarios, including the unlearning of sensitive features, backdoor features, and biased features. ### Experimental Results - **Sensitive Feature Unlearning**: Ferrari showed lower feature sensitivity and attack success rate (ASR) in sensitive feature unlearning scenarios and effectively prevented model inversion attacks (MIA) from reconstructing sensitive features. - **Backdoor Feature Unlearning**: Ferrari exhibited higher clean dataset accuracy (Accr) and lower backdoor dataset accuracy (Accu) in backdoor feature unlearning scenarios, indicating its ability to remove backdoor features while maintaining model utility. - **Biased Feature Unlearning**: Ferrari achieved similar accuracy between clean and biased datasets (Accr ≈ Accu) in biased feature unlearning scenarios, ensuring fairness and maintaining high overall accuracy. ### Conclusion By introducing feature sensitivity and the Ferrari framework, the paper addresses the challenges of feature unlearning in federated learning, providing a new method for efficient feature unlearning. This method is not only theoretically proven but also validated in multiple real-world scenarios, demonstrating its advantages in privacy protection and model utility.

Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity

Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics

Enable the Right to be Forgotten with Federated Client Unlearning in Medical Imaging

Forgettable Federated Linear Learning with Certified Data Unlearning

Unlearning during Learning: An Efficient Federated Machine Unlearning Method

A Survey on Federated Unlearning: Challenges, Methods, and Future Directions

Towards Federated Domain Unlearning: Verification Methodologies and Challenges

FedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models

A Survey of Federated Unlearning: A Taxonomy, Challenges and Future Directions

FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method

Communication Efficient and Provable Federated Unlearning

Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience

Subspace Based Federated Unlearning

Streamlined Federated Unlearning: Unite as One to Be Highly Efficient

Upcycling Noise for Federated Unlearning

The Right to Be Forgotten in Federated Learning: an Efficient Realization with Rapid Retraining

Server-Initiated Federated Unlearning to Eliminate Impacts of Low-Quality Data

Privacy-Preserving Federated Unlearning with Certified Client Removal

BFU: Bayesian Federated Unlearning with Parameter Self-Sharing.

An Accelerator of Feature Selection Applying a General Fuzzy Rough Model.