Abstract:Conventional synchronous federated learning (SFL) frameworks suffer from performance degradation in heterogeneous systems due to imbalanced local data size and diverse computing power on the client side. To address this problem, asynchronous FL (AFL) and semi-asynchronous FL have been proposed to recover the performance loss by allowing asynchronous aggregation. However, asynchronous aggregation incurs a new problem of inconsistency between local updates and global updates. Motivated by the issues of conventional SFL and AFL, we first propose a time-driven SFL (T-SFL) framework for heterogeneous systems. The core idea of T-SFL is that the server aggregates the models from different clients, each with varying numbers of iterations, at regular time intervals. To evaluate the learning performance of T-SFL, we provide an upper bound on the global loss function. Further, we optimize the aggregation weights to minimize the developed upper bound. Then, we develop a discriminative model selection (DMS) algorithm that removes local models from clients whose number of iterations falls below a predetermined threshold. In particular, this algorithm ensures that each client's aggregation weight accurately reflects its true contribution to the global model update, thereby improving the efficiency and robustness of the system. To validate the effectiveness of T-SFL with the DMS algorithm, we conduct extensive experiments using several popular datasets including MNIST, Cifar-10, Fashion-MNIST, and SVHN. The experimental results demonstrate that T-SFL with the DMS algorithm can reduce the latency of conventional SFL by 50\%, while achieving an average 3\% improvement in learning accuracy over state-of-the-art AFL algorithms.

MSSA-FL: High-Performance Multi-stage Semi-asynchronous Federated Learning with Non-IID Data.

FedSiam-DA: Dual-aggregated Federated Learning Via Siamese Network for Non-Iid Data

FedSA: A Semi-Asynchronous Federated Learning Mechanism in Heterogeneous Edge Computing

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework

Federated mutual learning

FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

Towards Efficient Asynchronous Federated Learning in Heterogeneous Edge Environments

Integrating Staleness and Shapley Value Consistency for Efficient K-Asynchronous Federated Learning

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

MDFL: Model-Distance Federated Learning on Non-IID Data

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

Edge-Assisted Hierarchical Federated Learning with Non-IID Data

Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

ASFL: Adaptive Semi-asynchronous Federated Learning for Balancing Model Accuracy and Total Latency in Mobile Edge Networks.

Adaptive Federated Learning on Non-IID Data with Resource Constraint

Confederated Learning: Federated Learning with Decentralized Edge Servers

Asynchronous Decentralized Federated Learning for Heterogeneous Devices