Abstract:As federated learning (FL) continues to advance, research has branched into two major directions: enhancing the individual global model and developing multiple personalized models. However, few studies took both orientations into consideration in parallel. The underlying reason lies in the objective heterogeneity between global and personalized models, where the former emphasizes generalization, while the latter focuses on specialization. The dilemma of sacrificing the personalized deviation in exchange for stable global enhancement, or forsaking the global model for personalization, is particularly acute in highly heterogeneous data scenarios (as a curse). To establish the coexistence of global and personalized models in highly heterogeneous data scenarios, we propose FedAKD, a generalized federated learning framework with adaptive knowledge distillation to optimizes both global and personalized models. Specifically, based on the similarity of predictive distributions over a reference dataset, two distinct adaptive weighting strategies are employed to match the divergent focuses of global and personalized optimization, leveraging data heterogeneity to foster cooperative adaptation and establish their coexistence (as a blessing). The client-side strategy through cluster-oriented optimization to facilitate the domain adaptation in local tasks. Meanwhile, the server-side strategy adjusts the global model to accommodate varying optimization directions from diverse implicit clusters. Experiments in label and feature shift settings demonstrate that our method outperforms state-of-the-art methods in both global and personalized performance with faster convergence [e.g. the accuracy improvements on the physical activity monitoring dataset (PAMAP2) for global and personalized models exceed 26.51% and 11.00%, respectively]. Furthermore, our method is equally applicable to real-world healthcare scenarios.

Personalized Federated Learning with Adaptive Batchnorm for Healthcare

FedHealth 2: Weighted Federated Transfer Learning via Batch Normalization for Personalized Healthcare

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

MetaFed: Federated Learning Among Federations With Cyclic Knowledge Distillation for Personalized Healthcare

Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

Personalized Federated Learning on Long-Tailed Data Via Adversarial Feature Augmentation.

Personalized Retrogress-Resilient Federated Learning Toward Imbalanced Medical Data

To Be Global or Personalized: Generalized Federated Learning with Cooperative Adaptation for Data Heterogeneity

Personalized Federated Learning via Feature Distribution Adaptation

FedMetaMed: Federated Meta-Learning for Personalized Medication in Distributed Healthcare Systems

FedAH: Aggregated Head for Personalized Federated Learning

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

Channel Adaptive and Sparsity Personalized Federated Learning for Privacy Protection in Smart Healthcare Systems

Group Personalized Federated Learning

Medical Federated Model with Mixture of Personalized and Shared Components

Medical Federated Model with Mixture of Personalized and Sharing Components

Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Sparse Federated Learning with Hierarchical Personalized Models