Abstract:Personalized federated learning (PFL) has gained increasing attention due to its success in handling the statistical heterogeneity of participants' local data by building distinct local models for participants. However, existing PFL schemes require the identical architecture and size of participants' models, e.g., the same number of layers in convolutional neural networks (CNN). In addition, the growing privacy issues (e.g., local update leakage to the curious server in model aggregation) have not been resolved in PFL. The utilization of identical model architectures among participants reduces the cost of privacy attacks since only one uniform attack method is required to extract private information, exacerbating the privacy threat. This paper proposes a novel privacy-preserving PFL framework that supports heterogeneous model architectures and sizes in delivering personalized models for different participants. Specifically, we utilize participants' knowledge, i.e., the soft predictions of local models on a public dataset, to effectively identify participants with similar data distributions regardless of the specific model architectures used. Based on the participants' knowledge, and their computing and storage capabilities, we employ the affinity propagation (AP) algorithm to implement a multi-level participant clustering mechanism for enabling heterogeneous PFL. Since knowledge is independent of original data, it is considered privacy-preserving during the clustering process. We also devise the ring aggregation algorithm to guarantee participants' privacy during the federated training process. In this way, each participant benefits from other participants with similar data distributions privately and obtains a satisfying personalized model. Furthermore, the cross-cluster knowledge transfer method boosts the personalization performance of weak participants. Sufficient theoretical analyses prove the effectiveness and privacy-preserving capacity of the proposed scheme. Extensive experiments on three benchmark datasets also demonstrate the superiority of our proposed scheme in various settings while maintaining privacy protection, outperforming other state-of-the-art schemes.

Privacy-preserving knowledge sharing for few-shot building energy prediction: A federated learning approach

Federated learning-based short-term building energy consumption prediction method for solving the data silos problem

Personalized federated learning for cross-building energy knowledge sharing: Cost-effective strategies and model architectures

Privacy-Preserving Power Consumption Prediction Based on Federated Learning with Cross-Entity Data

A Fair and Efficient Hybrid Federated Learning Framework based on XGBoost for Distributed Power Prediction

An adaptive federated learning system for community building energy load forecasting and anomaly prediction

Privacy preserved and decentralized thermal comfort prediction model for smart buildings using federated learning

Selective knowledge sharing for privacy-preserving federated distillation without a good teacher

Privacy-Preserving Heterogeneous Personalized Federated Learning with Knowledge

An Efficient Digital Twin Assisted Clustered Federated Learning Algorithm for Disease Prediction.

Personalized and privacy-enhanced federated learning framework via knowledge distillation

A privacy-preserving framework integrating federated learning and transfer learning for wind power forecasting

Differentially Private Knowledge Transfer for Federated Learning

PVFL: Verifiable Federated Learning and Prediction with Privacy-Preserving

FedRec: Privacy-Preserving News Recommendation with Federated Learning

Communication-efficient federated learning via knowledge distillation

Energy Prediction using Federated Learning

A Hybrid Federated Learning Framework with Dynamic Task Allocation for Multi-Party Distributed Load Prediction

Privacy-preserving Transfer Learning for Knowledge Sharing.

Federated Learning for Short-term Residential Load Forecasting