Abstract:Federated learning (FL) is a popular approach to facilitate privacy-aware machine learning since it allows multiple clients to collaboratively train a global model without granting others access to their private data. It is, however, known that FL can be vulnerable to membership inference attacks (MIAs), where the training records of the global model can be distinguished from the testing records. Surprisingly, research focusing on the investigation of the source inference problem appears to be lacking. We also observe that identifying a training record's source client can result in privacy breaches extending beyond MIAs. For example, consider an FL application where multiple hospitals jointly train a COVID-19 diagnosis model, membership inference attackers can identify the medical records that have been used for training, and any additional identification of the source hospital can result the patient from the particular hospital more prone to discrimination. Seeking to contribute to the literature gap, we take the first step to investigate source privacy in FL. Specifically, we propose a new inference attack (hereafter referred to as source inference attack – SIA), designed to facilitate an honest-but-curious server to identify the training record's source client. The proposed SIAs leverage the Bayesian theorem to allow the server to implement the attack in a non-intrusive manner without deviating from the defined FL protocol. We then evaluate SIAs in three different FL frameworks to show that in existing FL frameworks, the clients sharing gradients, model parameters, or predictions on a public dataset will leak such source information to the server. We also conduct extensive experiments on various datasets to investigate the key factors in an SIA. The experimental results validate the efficacy of the proposed SIAs, e.g., an attack success rate of 67.1% (baseline 10%) can be achieved when the clients share model parameters with the server. Comprehensive ablation studies demonstrate that the success of an SIA is directly related to the overfitting of the local models.

Accuracy-Privacy Trade-off in the Mitigation of Membership Inference Attack in Federated Learning

MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learning

FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learning

Addressing Membership Inference Attack in Federated Learning with Model Compression

Practical Private Aggregation in Federated Learning Against Inference Attack

CS-MIA: Membership Inference Attack Based on Prediction Confidence Series in Federated Learning

Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspective

MemberShield: A framework for federated learning with membership privacy

Source Inference Attacks in Federated Learning

Subject-Level Membership Inference Attack via Data Augmentation and Model Discrepancy

Benchmarking Robustness and Privacy-Preserving Methods in Federated Learning

Subject Membership Inference Attacks in Federated Learning

Privacy Attack in Federated Learning is Not Easy: An Experimental Study

Toward the Tradeoffs between Privacy, Fairness and Utility in Federated Learning

Source Inference Attacks: Beyond Membership Inference Attacks in Federated Learning

A New Implementation of Federated Learning for Privacy and Security Enhancement

A Multi-Shuffler Framework to Establish Mutual Confidence for Secure Federated Learning

Active Membership Inference Attack under Local Differential Privacy in Federated Learning

Federated Learning Privacy: Attacks, Defenses, Applications, and Policy Landscape - A Survey

Efficient, Private and Robust Federated Learning