Abstract:Federated learning (FL) is an efficient decentralized machine learning methodology for processing nonindependent and identically distributed (non-IID) data due to geographical and temporal distribution differences. Non-IID data generally indicates substantial disparities in data distribution and features among clients. This assumption is completely different from the conventional assumption of independent and identically distributed (IID) data in which all clients' data originates from the same distribution. There are many factors that affect the features of non-IID data, such as user preferences, data collection methods, and client characteristics. The factors of data distribution, category proportions, and feature representation also affect the statistical properties of non-IID data. This article conducts an in-depth exploration of FL with the consideration of diverse features and statistical properties of non-IID data. Specifically, we first discuss the impact of non-IID data on communication efficiency, model convergence, and FL accuracy. The presence of non-IID data leads to increased communication overhead, imbalanced class distribution, and uneven local model updates. All of these affect FL convergence and performance. Then, we present the latest advanced techniques, such as data partitioning/sharing, client selection, differential privacy, and secure aggregation, which are used to address the challenges posed by non-IID data in terms of communication efficiency and privacy protection. Furthermore, we show the emerging applications and use cases of FL with non-IID data in various domains, such as healthcare, Internet of Things, and edge computing. Overall, this survey provides a comprehensive understanding of FL with non-IID data, including the challenges, advancements, and practical applications in different areas.

Enhanced Federated Learning on Non-Iid Data Via Local Importance Sampling

ISFL: Federated Learning for Non-i.i.d. Data with Local Importance Sampling

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Non-IID always Bad? Semi-Supervised Heterogeneous Federated Learning with Local Knowledge Enhancement

Fed-FSNet: Mitigating Non-I.I.D. Federated Learning via Fuzzy Synthesizing Network

FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distribution

FedEL: Federated ensemble learning for non-iid data

A Survey of Federated Learning on Non-IID Data

Federated Learning with Non-IID Data: A Survey

One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversity

Privacy-Preserving Federated Learning Against Label-Flipping Attacks on Non-IID Data

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

Federated Learning on Non-IID Data Silos: An Experimental Study

FedSampling: A Better Sampling Strategy for Federated Learning

Privacy-Preserving Asynchronous Federated Learning Under Non-IID Settings

AggEnhance: Aggregation Enhancement by Class Interior Points in Federated Learning with Non-IID Data.

FLAS: Computation and Communication Efficient Federated Learning via Adaptive Sampling

GFL: Federated Learning on Non-IID data via Privacy-preserving Synthetic data

FedSLD: Federated Learning with Shared Label Distribution for Medical Image Classification

FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data.

Communication-efficient federated continual learning for distributed learning system with Non-IID data