Abstract:Personalized federated learning (PFL) has gained increasing attention due to its success in handling the statistical heterogeneity of participants' local data by building distinct local models for participants. However, existing PFL schemes require the identical architecture and size of participants' models, e.g., the same number of layers in convolutional neural networks (CNN). In addition, the growing privacy issues (e.g., local update leakage to the curious server in model aggregation) have not been resolved in PFL. The utilization of identical model architectures among participants reduces the cost of privacy attacks since only one uniform attack method is required to extract private information, exacerbating the privacy threat. This paper proposes a novel privacy-preserving PFL framework that supports heterogeneous model architectures and sizes in delivering personalized models for different participants. Specifically, we utilize participants' knowledge, i.e., the soft predictions of local models on a public dataset, to effectively identify participants with similar data distributions regardless of the specific model architectures used. Based on the participants' knowledge, and their computing and storage capabilities, we employ the affinity propagation (AP) algorithm to implement a multi-level participant clustering mechanism for enabling heterogeneous PFL. Since knowledge is independent of original data, it is considered privacy-preserving during the clustering process. We also devise the ring aggregation algorithm to guarantee participants' privacy during the federated training process. In this way, each participant benefits from other participants with similar data distributions privately and obtains a satisfying personalized model. Furthermore, the cross-cluster knowledge transfer method boosts the personalization performance of weak participants. Sufficient theoretical analyses prove the effectiveness and privacy-preserving capacity of the proposed scheme. Extensive experiments on three benchmark datasets also demonstrate the superiority of our proposed scheme in various settings while maintaining privacy protection, outperforming other state-of-the-art schemes.

Privacy-Preserving Federated Learning Framework with General Aggregation and Multiparty Entity Matching

EPFed: Achieving Optimal Balance between Privacy and Efficiency in Federated Learning

Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing

Secure Decentralized Aggregation to Prevent Membership Privacy Leakage in Edge-based Federated Learning

DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting

Privacy-Preserving Model Aggregation for Asynchronous Federated Learning

A privacy preserving federated learning scheme using homomorphic encryption and secret sharing

A Security-Enhanced Federated Learning Scheme Based on Homomorphic Encryption and Secret Sharing

An Efficient and Multi-private Key Secure Aggregation for Federated Learning

Privacy-Enhanced and Verification-Traceable Aggregation for Federated Learning.

Efficient secure aggregation for privacy-preserving federated learning based on secret sharing

Efficient Vertical Federated Learning with Secure Aggregation

Privacy-Preserving Heterogeneous Personalized Federated Learning with Knowledge

PFLM: Privacy-preserving federated learning with membership proof

LF3PFL: A Practical Privacy-Preserving Federated Learning Algorithm Based on Local Federalization Scheme

An Efficient and Multi-Private Key Secure Aggregation Scheme for Federated Learning

Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning

Privacy-preserving Decentralized Aggregation for Federated Learning

PVD-FL: A Privacy-Preserving and Verifiable Decentralized Federated Learning Framework

EFMVFL: An Efficient and Flexible Multi-party Vertical Federated Learning without a Third Party

Practical Private Aggregation in Federated Learning Against Inference Attack