Abstract:Federated learning (FL) is an emerging, privacy-preserving machine learning paradigm, drawing tremendous attention in both academia and industry. A unique characteristic of FL is heterogeneity, which resides in the various hardware specifications and dynamic states across the participating devices. Theoretically, heterogeneity can exert a huge influence on the FL training process, e.g., causing a device unavailable for training or unable to upload its model updates. Unfortunately, these impacts have never been systematically studied and quantified in existing FL literature. In this paper, we carry out the first empirical study to characterize the impacts of heterogeneity in FL. We collect large-scale data from 136k smartphones that can faithfully reflect heterogeneity in real-world settings. We also build a heterogeneity-aware FL platform that complies with the standard FL protocol but with heterogeneity in consideration. Based on the data and the platform, we conduct extensive experiments to compare the performance of state-of-the-art FL algorithms under heterogeneity-aware and heterogeneity-unaware settings. Results show that heterogeneity causes non-trivial performance degradation in FL, including up to 9.2% accuracy drop, 2.32 × lengthened training time, and undermined fairness. Furthermore, we analyze potential impact factors and find that device failure and participant bias are two potential factors for performance degradation. Our study provides insightful implications for FL practitioners. On the one hand, our findings suggest that FL algorithm designers consider necessary heterogeneity during the evaluation. On the other hand, our findings urge system providers to design specific mechanisms to mitigate the impacts of heterogeneity.

FLASH: Heterogeneity-Aware Federated Learning at Scale

Hierarchical Federated Learning: the Interplay of User Mobility and Data Heterogeneity

Characterizing Impacts of Heterogeneity in Federated Learning Upon Large-Scale Smartphone Data

FLASH: Federated Learning Across Simultaneous Heterogeneities

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Clustered Federated Learning in Heterogeneous Environment.

AdaptFL: Adaptive Federated Learning Framework for Heterogeneous Devices

Heterogeneous Federated Learning: State-of-the-art and Research Challenges

Asynchronous Federated Learning on Heterogeneous Devices: A Survey

A Survey on Heterogeneous Federated Learning

Heterogeneity-Aware Federated Learning with Adaptive Local Epoch Size in Edge Computing

ParallelSFL: A Novel Split Federated Learning Framework Tackling Heterogeneity Issues

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Advances in Robust Federated Learning: Heterogeneity Considerations

HeteroSwitch: Characterizing and Taming System-Induced Data Heterogeneity in Federated Learning

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

Equalized Aggregation for Heterogeneous Federated Mobile Edge Learning

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation

Completely Heterogeneous Federated Learning

HADFL: Heterogeneity-aware Decentralized Federated Learning Framework

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity