Abstract:Federated learning (FL) has enabled collaborative model training across decentralized data sources or clients. While adding new participants to a shared model does not pose great technical hurdles, the removal of a participant and their related information contained in the shared model remains a challenge. To address this problem, federated unlearning has emerged as a critical research direction, seeking to remove information from globally trained models without harming the model performance on the remaining data. Most modern federated unlearning methods use costly approaches such as the use of remaining clients data to retrain the global model or methods that would require heavy computation on client or server side. We introduce Contribution Dampening (ConDa), a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client and performs synaptic dampening on the parameters of the global model that have privacy infringing contributions from the forgetting client. Our technique does not require clients data or any kind of retraining and it does not put any computational overhead on either the client or server side. We perform experiments on multiple datasets and demonstrate that ConDa is effective to forget a client's data. In experiments conducted on the MNIST, CIFAR10, and CIFAR100 datasets, ConDa proves to be the fastest federated unlearning method, outperforming the nearest state of the art approach by at least 100x. Our emphasis is on the non-IID Federated Learning setting, which presents the greatest challenge for unlearning. Additionally, we validate ConDa's robustness through backdoor and membership inference attacks. We envision this work as a crucial component for FL in adhering to legal and ethical requirements.

What problem does this paper attempt to address?

The paper attempts to address the problem of how to efficiently and effectively remove the data contribution of specific clients in Federated Learning (FL), known as Federated Unlearning. Specifically, when a client requests the deletion of their data, how to remove the influence of that client's data from the global model without retraining the model or affecting the data of other clients, while maintaining the overall performance of the model. ### Background and Challenges 1. **Background of Federated Learning**: - Federated Learning allows multiple clients to collaboratively train a machine learning model without sharing data, thereby improving the accuracy and robustness of the model. - Each client updates the model parameters through local training and then sends these updates to a central server for aggregation, forming a global model. 2. **Need for Federated Unlearning**: - Clients may request the deletion of their data contribution due to contractual, legal compliance, or privacy reasons. - The global model may inadvertently remember information from individual data sources, necessitating a method to remove this information. 3. **Limitations of Existing Methods**: - Most existing federated unlearning methods rely on retraining the model or using the remaining clients' data, which is not only computationally expensive but may also infringe on the privacy of other clients. - It is unrealistic to assume that the remaining clients still hold the training data when retraining or updating the model. - Federated unlearning is more complex in non-independent and identically distributed (non-IID) data settings because the data distribution among different clients is uneven, and removing one client's data may affect the model's learning effectiveness on other clients' data. ### Main Contributions of the Paper 1. **CONDA Framework**: - Proposes a framework called CONDA (Contribution Dampening), which achieves efficient federated unlearning by tracking each client's influence parameters on the global model and selectively suppressing these parameters. - CONDA does not require retraining the model or accessing the remaining clients' data, thereby reducing computational overhead and protecting client privacy. 2. **Data-Independent and Efficient Unlearning**: - CONDA can effectively remove the data contribution of specific clients without retraining the model, while maintaining the overall performance of the model. - Experimental validation shows that CONDA performs excellently on multiple datasets, being at least 100 times faster than existing methods. 3. **Experimental Validation**: - Experiments on MNIST, CIFAR-10, and CIFAR-100 datasets demonstrate the effectiveness of CONDA. - The experimental results show that CONDA can maintain the model's performance on retained data while removing client data and exhibits good privacy protection capabilities in backdoor attacks and membership inference attacks. ### Conclusion The paper proposes a new federated unlearning method, CONDA, which achieves efficient and privacy-preserving federated unlearning through selective parameter suppression. CONDA not only significantly outperforms existing methods in computational efficiency but also excels in model performance and privacy protection, making it particularly suitable for non-independent and identically distributed (non-IID) federated learning environments.

ConDa: Fast Federated Unlearning with Contribution Dampening

Subspace Based Federated Unlearning

DCFL: Non-IID awareness Data Condensation aided Federated Learning

Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience

Continual Federated Learning Based on Knowledge Distillation

Forgettable Federated Linear Learning with Certified Data Unlearning

Federated Unlearning via Active Forgetting

Communication Efficient and Provable Federated Unlearning

DSFedCon: Dynamic Sparse Federated Contrastive Learning for Data-Driven Intelligent Systems

Efficient Federated Unlearning under Plausible Deniability

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics

FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teacher

Acceleration of Federated Learning with Alleviated Forgetting in Local Training

Collaboratively Learning Federated Models from Noisy Decentralized Data

Unlearning during Learning: An Efficient Federated Machine Unlearning Method

On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing

FedMD: Heterogenous Federated Learning via Model Distillation

Accurate Forgetting for Heterogeneous Federated Continual Learning

FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning

Knowledge Lock: Overcoming Catastrophic Forgetting in Federated Learning