Abstract:Personalized federated learning (PFL) has gained increasing attention due to its success in handling the statistical heterogeneity of participants' local data by building distinct local models for participants. However, existing PFL schemes require the identical architecture and size of participants' models, e.g., the same number of layers in convolutional neural networks (CNN). In addition, the growing privacy issues (e.g., local update leakage to the curious server in model aggregation) have not been resolved in PFL. The utilization of identical model architectures among participants reduces the cost of privacy attacks since only one uniform attack method is required to extract private information, exacerbating the privacy threat. This paper proposes a novel privacy-preserving PFL framework that supports heterogeneous model architectures and sizes in delivering personalized models for different participants. Specifically, we utilize participants' knowledge, i.e., the soft predictions of local models on a public dataset, to effectively identify participants with similar data distributions regardless of the specific model architectures used. Based on the participants' knowledge, and their computing and storage capabilities, we employ the affinity propagation (AP) algorithm to implement a multi-level participant clustering mechanism for enabling heterogeneous PFL. Since knowledge is independent of original data, it is considered privacy-preserving during the clustering process. We also devise the ring aggregation algorithm to guarantee participants' privacy during the federated training process. In this way, each participant benefits from other participants with similar data distributions privately and obtains a satisfying personalized model. Furthermore, the cross-cluster knowledge transfer method boosts the personalization performance of weak participants. Sufficient theoretical analyses prove the effectiveness and privacy-preserving capacity of the proposed scheme. Extensive experiments on three benchmark datasets also demonstrate the superiority of our proposed scheme in various settings while maintaining privacy protection, outperforming other state-of-the-art schemes.

Exploiting Complex Network-based Clustering for Personalization-Enhanced Hierarchical Federated Edge Learning

Edge-cloud Collaborative Learning with Federated and Centralized Features

Hierarchical Federated Learning: the Interplay of User Mobility and Data Heterogeneity

Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networks

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Enhanced Hybrid Hierarchical Federated Edge Learning Over Heterogeneous Networks

Optimizing Efficient Personalized Federated Learning with Hypernetworks at Edge

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

Hierarchical Federated Edge Learning with Adaptive Clustering in Internet of Things

A Communication-Efficient Hierarchical Federated Learning Framework via Shaping Data Distribution at Edge

Optimizing Training Efficiency and Cost of Hierarchical Federated Learning in Heterogeneous Mobile-Edge Cloud Computing

Robust Hierarchical Federated Learning with Anomaly Detection in Cloud-Edge-End Cooperation Networks

Over-the-Air Hierarchical Personalized Federated Learning

Federated Learning as a Service for Hierarchical Edge Networks with Heterogeneous Models

Sparse Federated Learning with Hierarchical Personalized Models

Privacy-Preserving Heterogeneous Personalized Federated Learning with Knowledge

Energy-Aware Edge Association for Cluster-based Personalized Federated Learning

Personalized federated learning for heterogeneous data: A distributed edge clustering approach

H-FedSN: Personalized Sparse Networks for Efficient and Accurate Hierarchical Federated Learning for IoT Applications

SHARE: Shaping Data Distribution at Edge for Communication-Efficient Hierarchical Federated Learning

High-Precision Cluster Federated Learning for Smart Home: An Edge-Cloud Collaboration Approach