Abstract:Federated Learning (FL) has gained considerable attention recently, as it allows clients to cooperatively train a global machine learning model without sharing raw data. However, its performance can be compromised due to the high heterogeneity in clients' local data distributions, commonly known as Non-IID (non-independent and identically distributed). Moreover, collaboration among highly dissimilar clients exacerbates this performance degradation. Personalized FL seeks to mitigate this by enabling clients to collaborate primarily with others who have similar data characteristics, thereby producing personalized models. We noticed that existing methods for assessing model similarity often do not capture the genuine relevance of client domains. In response, our paper enhances personalized client collaboration in FL by introducing a metric for domain relevance between clients. Specifically, to facilitate optimal coalition formation, we measure the marginal contributions of client models using coalition game theory, providing a more accurate representation of potential client domain relevance within the FL privacy-preserving framework. Based on this metric, we then adjust each client's coalition membership and implement a personalized FL aggregation algorithm that is robust to Non-IID data domain. We provide a theoretical analysis of the algorithm's convergence and generalization capabilities. Our extensive evaluations on multiple datasets, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100, and under varying Non-IID data distributions (Pathological and Dirichlet), demonstrate that our personalized collaboration approach consistently outperforms contemporary benchmarks in terms of accuracy for individual clients.

FedMCSA: Personalized federated learning via model components self-attention

Personalized Federated Learning with Local Attention

Personalized Federated Learning with Attention-based Client Selection

Personalized federated learning based on multi-head attention algorithm

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning

Client-supervised Federated Learning: Towards One-model-for-all Personalization

Bilateral Improvement in Local Personalization and Global Generalization in Federated Learning.

Rethinking Personalized Client Collaboration in Federated Learning

APFed: Attention-based Personalized Federated Learning in Model Heterogeneity Scenario.

Improving Generalization and Personalization in Model-Heterogeneous Federated Learning

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

Pfedsim: Similarity-Aware Model Aggregation Towards Personalized Federated Learning.

ChannelFed: Enabling Personalized Federated Learning via Localized Channel Attention

FedCR: Personalized Federated Learning Based on Across-Client Common Representation with Conditional Mutual Information Regularization.

Federated mutual learning

Personalized Federated Learning: an Attentive Collaboration Approach

Feed: Towards Personalization-Effective Federated Learning

A personalized federated learning method based on the residual multi-head attention mechanism

FedCML: Federated Clustering Mutual Learning with non-IID Data.

FedCE: Personalized Federated Learning Method based on Clustering Ensembles