Abstract:Federated learning (FL) is a distributed machine learning paradigm under privacy preservation. However, data heterogeneity among clients leads to the shared global model obtained after training, which cannot fit the distribution of each client's dataset, and the performance of the model degrades. To address this problem, we proposed a personalized federated learning method based on clustering and knowledge distillation, called pFedCK. In this algorithm, each client has an interactive model that participates in global training and a personalized model that is only trained locally. Both of the models perform knowledge distillation with each other through the feature representation of the middle layer and the soft prediction of the model. In addition, in order to make an interaction model only obtaining the model information from the client, which has similar data distribution and avoids the interference of other heterogeneous information, the server will cluster the clients according to the similarity of the amount of parameter variation uploaded by different interaction models during every training round. By clustering clients, interaction models with similar data distributions can cooperate with each other to better fit the local dataset distribution. Thereby, the performance of personalized model can be improved by obtaining more valuable information indirectly. Finally, we conduct simulation experiments on three benchmark datasets under different data heterogeneity scenarios. Compared to the single model algorithms, the accuracy of pFedCK improved by an average of 23.4% and 23.8% over FedAvg and FedProx, respectively; compared to typical personalization algorithms, the accuracy of pFedCK improved by an average of 0.8% and 1.3%, and a maximum of 1.0% and 2.9% over FedDistill and FML.

MetaFed: Federated Learning Among Federations With Cyclic Knowledge Distillation for Personalized Healthcare

FedMetaMed: Federated Meta-Learning for Personalized Medication in Distributed Healthcare Systems

FedDGP: Disentangling Global and Personal Models for Federated Learning

A Federated Learning Framework Via Decentralized Data Valuation for Chronic Disease Healthcare

A Personalized Federated Learning Method Based on Clustering and Knowledge Distillation

Personalized Federated Learning with Adaptive Batchnorm for Healthcare

Tailored Federated Learning: Leveraging Direction Regulation & Knowledge Distillation

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

MH-pFLID: Model Heterogeneous Personalized Federated Learning Via Injection and Distillation for Medical Data Analysis

A personalized federated meta-learning method for intelligent and privacy-preserving fault diagnosis

Meta-FL: A Novel Meta-Learning Framework for Optimizing Heterogeneous Model Aggregation in Federated Learning

Towards Personalized Federated Learning via Comprehensive Knowledge Distillation

Sparse Federated Learning with Hierarchical Personalized Models

FedHealth 2: Weighted Federated Transfer Learning via Batch Normalization for Personalized Healthcare

Medical Federated Model with Mixture of Personalized and Sharing Components

Handling Data Heterogeneity in Federated Learning via Knowledge Distillation and Fusion

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Medical Federated Model with Mixture of Personalized and Shared Components

FedL2P: Federated Learning to Personalize

PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records

FedD2S: Personalized Data-Free Federated Knowledge Distillation