Abstract:Federated Learning (FL) is a distributed machine learning paradigm that achieves a globally robust model through decentralized computation and periodic model synthesis, primarily focusing on the global model's accuracy over aggregated datasets of all participating clients. Personalized Federated Learning (PFL) instead tailors exclusive models for each client, aiming to enhance the accuracy of clients' individual models on specific local data distributions. Despite of their wide adoption, existing FL and PFL works have yet to comprehensively address the class-imbalance issue, one of the most critical challenges within the realm of data heterogeneity in PFL and FL research. In this paper, we propose FedReMa, an efficient PFL algorithm that can tackle class-imbalance by 1) utilizing an adaptive inter-client co-learning approach to identify and harness different clients' expertise on different data classes throughout various phases of the training process, and 2) employing distinct aggregation methods for clients' feature extractors and classifiers, with the choices informed by the different roles and implications of these model components. Specifically, driven by our experimental findings on inter-client similarity dynamics, we develop critical co-learning period (CCP), wherein we introduce a module named maximum difference segmentation (MDS) to assess and manage task relevance by analyzing the similarities between clients' logits of their classifiers. Outside the CCP, we employ an additional scheme for model aggregation that utilizes historical records of each client's most relevant peers to further enhance the personalization stability. We demonstrate the superiority of our FedReMa in extensive experiments.

Robust Multi-model Personalized Federated Learning Via Model Distillation

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedMD: Heterogenous Federated Learning via Model Distillation

Decentralized Federated Learning Via Mutual Knowledge Distillation.

Robust Federated Learning for Heterogeneous Model and Data

A Novel Bayesian Federated Learning Framework to Address Multi-Dimensional Heterogeneity Problem

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

Ensemble Distillation for Robust Model Fusion in Federated Learning

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

Multi-Dimensional Fair Federated Learning

Federated Mutual Learning: a Collaborative Machine Learning Method for Heterogeneous Data, Models, and Objectives

DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning

Personalized and privacy-enhanced federated learning framework via knowledge distillation

Confidence-aware Personalized Federated Learning via Variational Expectation Maximization

MetaFed: Federated Learning Among Federations With Cyclic Knowledge Distillation for Personalized Healthcare

Unifying Distillation with Personalization in Federated Learning

Dual Calibration-based Personalised Federated Learning

Federated Multi-Task Learning under a Mixture of Distributions

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Sparse Personalized Federated Learning

MDFL: Model-Distance Federated Learning on Non-IID Data