Abstract:Federated Learning (FL) is a novel distributed machine learning, which allows thousands of edge devices to train models locally without uploading data to the central server. Since devices in real federated settings are resource-constrained, FL encounters systems heterogeneity, which causes considerable stragglers and incurs significant accuracy degradation. To tackle the challenges of systems heterogeneity and improve the robustness of the global model, we propose a novel adaptive federated framework in this paper. Specifically, we propose FedSAE that leverages the workload completion history of clients to adaptively predict the affordable training workload for each device. Consequently, FedSAE can significantly reduce stragglers in highly heterogeneous systems. We incorporate Active Learning into FedSAE to dynamically schedule participants. The server evaluates the devices' training value based on their training loss in each round, and larger-value clients are selected with a higher probability. As a result, the model convergence is accelerated. Furthermore, we propose q-FedSAE that combines FedSAE and q-FFL to improve global fairness in highly heterogeneous systems. The evaluations conducted in a highly heterogeneous system demonstrate that both FedSAE and q-FedSAE converge faster than FedAvg. In particular, FedSAE outperforms FedAvg across multiple federated datasets - FedSAE improves testing accuracy by 22.19% and reduces stragglers by 90.69% on average. Moreover, holding the same accuracy as FedSAE, q-FedSAE allows for more robust convergence and fairer model performance than q-FedAvg, FedSAE.

Tackling Feature Skew in Heterogeneous Federated Learning with Semantic Enhancement

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

FedSeg: Class-Heterogeneous Federated Learning for Semantic Segmentation

Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversity

The Key of Parameter Skew in Federated Learning

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Towards Addressing Heterogeneity Of Data In Federated Learning

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Federated learning with workload-aware client scheduling in heterogeneous systems

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

FED-SDS: Adaptive Structured Dynamic Sparsity for Federated Learning under Heterogeneous Clients

FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning

Personalized federated learning based on feature fusion

FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation

FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning

A Simple Data Augmentation for Feature Distribution Skewed Federated Learning