Abstract:Federated Learning (FL) is a rising approach towards collaborative and privacy-preserving machine learning where large-scale medical datasets remain localized to each client. However, the issue of data heterogeneity among clients often compels local models to diverge, leading to suboptimal global models. To mitigate the impact of data heterogeneity on FL performance, we start with analyzing how FL training influence FL performance by decomposing the global loss into three terms: local loss, distribution shift loss and aggregation loss. Remarkably, our loss decomposition reveals that existing local training-based FL methods attempt to reduce the distribution shift loss, while the global aggregation-based FL methods propose better aggregation strategies to reduce the aggregation loss. Nevertheless, a comprehensive joint effort to minimize all three terms is currently limited in the literature, leading to subpar performance when dealing with data heterogeneity challenges. To fill this gap, we propose a novel FL method based on global loss decomposition, called FedLD, to jointly reduce these three loss terms. Our FedLD involves a margin control regularization in local training to reduce the distribution shift loss, and a principal gradient-based server aggregation strategy to reduce the aggregation loss. Notably, under different levels of data heterogeneity, our strategies achieve better and more robust performance on retinal and chest X-ray classification compared to other FL algorithms. Our code is available at <a class="link-external link-https" href="https://github.com/Zeng-Shuang/FedLD" rel="external noopener nofollow">this https URL</a>.

Towards Addressing Heterogeneity Of Data In Federated Learning

Hierarchical Federated Learning: the Interplay of User Mobility and Data Heterogeneity

Advances in Robust Federated Learning: Heterogeneity Considerations

Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning

Virtual Homogeneity Learning: Defending Against Data Heterogeneity in Federated Learning

Privacy preserving federated learning for full heterogeneity

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Completely Heterogeneous Federated Learning

Addressing Heterogeneity in Federated Learning: Challenges and Solutions for a Shared Production Environment

A Survey on Heterogeneous Federated Learning

Tackling Data Heterogeneity in Federated Learning via Loss Decomposition

FedH2L: A Federated Learning Approach with Model and Statistical Heterogeneity

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

Clustered Federated Learning in Heterogeneous Environment.

A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Heterogeneous Federated Learning: State-of-the-art and Research Challenges

An Efficient and Security Federated Learning for Data Heterogeneity

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity

Handling Data Heterogeneity in Federated Learning via Knowledge Distillation and Fusion

Heterogeneity-Aware Federated Learning with Adaptive Local Epoch Size in Edge Computing