Abstract:Federated learning (FL) is an emerging, privacy-preserving machine learning paradigm, drawing tremendous attention in both academia and industry. A unique characteristic of FL is heterogeneity, which resides in the various hardware specifications and dynamic states across the participating devices. Theoretically, heterogeneity can exert a huge influence on the FL training process, e.g., causing a device unavailable for training or unable to upload its model updates. Unfortunately, these impacts have never been systematically studied and quantified in existing FL literature. In this paper, we carry out the first empirical study to characterize the impacts of heterogeneity in FL. We collect large-scale data from 136k smartphones that can faithfully reflect heterogeneity in real-world settings. We also build a heterogeneity-aware FL platform that complies with the standard FL protocol but with heterogeneity in consideration. Based on the data and the platform, we conduct extensive experiments to compare the performance of state-of-the-art FL algorithms under heterogeneity-aware and heterogeneity-unaware settings. Results show that heterogeneity causes non-trivial performance degradation in FL, including up to 9.2% accuracy drop, 2.32 × lengthened training time, and undermined fairness. Furthermore, we analyze potential impact factors and find that device failure and participant bias are two potential factors for performance degradation. Our study provides insightful implications for FL practitioners. On the one hand, our findings suggest that FL algorithm designers consider necessary heterogeneity during the evaluation. On the other hand, our findings urge system providers to design specific mechanisms to mitigate the impacts of heterogeneity.

Privacy-Preserving Federated Learning for Data Heterogeneity in 6G Mobile Networks

Hierarchical Federated Learning: the Interplay of User Mobility and Data Heterogeneity

An Improved Federated Learning Algorithm for Privacy Preserving in Cybertwin-Driven 6G System

Optimization Design for Federated Learning in Heterogeneous 6G Networks

A Novel Federated Learning Framework Based on Conditional Generative Adversarial Networks for Privacy Preserving in 6G

Federated Learning with Unsourced Random Access.

Security and Privacy in Artificial Intelligence-Enabled 6G

Federated Learning for 6G: Applications, Challenges, and Opportunities

Security and Privacy of 6G Federated Learning-enabled Dynamic Spectrum Sharing

Towards Energy Efficient Distributed Federated Learning for 6G Networks

Federated Learning for 6G: Paradigms, Taxonomy, Recent Advances and Insights

FLPP: A Federated-Learning-Based Scheme for Privacy Protection in Mobile Edge Computing

Federated learning for 6G communications: Challenges, methods, and future directions

Lightweight Privacy-Preserving Cross-Cluster Federated Learning with Heterogeneous Data

Efficient Hierarchical Federated Services for Heterogeneous Mobile Edge

A Secure Federated Learning Framework for 5G Networks

Toward 6G Networks: Architecture and Strategy for Information Security Transmission Based on Multi-Source Heterogeneous Data

Decentralized P2P Federated Learning for Privacy-Preserving and Resilient Mobile Robotic Systems

FedCST: Federated Learning on Heterogeneous Resource-constrained Devices Using Clustering and Split Training

Characterizing Impacts of Heterogeneity in Federated Learning Upon Large-Scale Smartphone Data

A Practical Cross-Device Federated Learning Framework over 5G Networks