Federated Unlearning via Active Forgetting

Yuyuan Li,Chaochao Chen,Xiaolin Zheng,Jiaming Zhang

2023-07-07

Abstract:The increasing concerns regarding the privacy of machine learning models have catalyzed the exploration of machine unlearning, i.e., a process that removes the influence of training data on machine learning models. This concern also arises in the realm of federated learning, prompting researchers to address the federated unlearning problem. However, federated unlearning remains challenging. Existing unlearning methods can be broadly categorized into two approaches, i.e., exact unlearning and approximate unlearning. Firstly, implementing exact unlearning, which typically relies on the partition-aggregation framework, in a distributed manner does not improve time efficiency theoretically. Secondly, existing federated (approximate) unlearning methods suffer from imprecise data influence estimation, significant computational burden, or both. To this end, we propose a novel federated unlearning framework based on incremental learning, which is independent of specific models and federated settings. Our framework differs from existing federated unlearning methods that rely on approximate retraining or data influence estimation. Instead, we leverage new memories to overwrite old ones, imitating the process of \textit{active forgetting} in neurology. Specifically, the model, intended to unlearn, serves as a student model that continuously learns from randomly initiated teacher models. To preserve catastrophic forgetting of non-target data, we utilize elastic weight consolidation to elastically constrain weight change. Extensive experiments on three benchmark datasets demonstrate the efficiency and effectiveness of our proposed method. The result of backdoor attacks demonstrates that our proposed method achieves satisfying completeness.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the issue of Federated Unlearning in machine learning models. Specifically, the paper proposes a new federated unlearning framework called FedAF, which is based on the concept of Active Forgetting. This framework aims to overcome several major challenges in existing federated unlearning methods:

1. **Inaccurate Data Impact Estimation**: Existing federated unlearning methods have inaccuracies in estimating the impact of data on the model, which affects the completeness of unlearning and the effectiveness of the model.
2. **Significant Computational Burden**: Many existing methods require substantial computational resources for federated unlearning, especially when dealing with large-scale datasets.
3. **Limitations of Existing Methods**: Existing federated unlearning methods either rely on approximate retraining (such as reverse unlearning) or data impact estimation, both of which have various issues in practical applications.

The FedAF framework proposed in the paper addresses the above issues in the following ways:
- **Generating New Memories**: It uses newly generated, knowledge-agnostic, and easy-to-learn memories to overwrite old memories, thereby achieving the unlearning of target data.
- **Overcoming Catastrophic Forgetting**: It introduces Elastic Weight Consolidation (EWC) to mitigate the loss of knowledge from non-target data.
- **Seamless Integration into the Federated Learning Process**: FedAF seamlessly integrates the unlearning process into the federated learning process, making it independent of specific models and settings, and reducing additional storage and computational overhead.

Overall, FedAF aims to improve the efficiency, effectiveness, and completeness of federated unlearning while maintaining model performance and protecting user privacy.