Abstract:Federated Learning (FL) has emerged as a powerful paradigm for decentralized machine learning, enabling collaborative model training across diverse clients without sharing raw data. However, traditional FL approaches often face limitations in scalability and efficiency due to their reliance on synchronous client updates, which can result in significant delays and increased communication overhead, particularly in heterogeneous and dynamic environments. To address these challenges in this paper, we propose an Asynchronous Federated Learning (AFL) algorithm, which allows clients to update the global model independently and asynchronously. Our key contributions include a comprehensive convergence analysis of AFL in the presence of client delays and model staleness. By leveraging martingale difference sequence theory and variance bounds, we ensure robust convergence despite asynchronous updates. Assuming strongly convex local objective functions, we establish bounds on gradient variance under random client sampling and derive a recursion formula quantifying the impact of client delays on convergence. Furthermore, we demonstrate the practical applicability of AFL by training a decentralized Long Short-Term Memory (LSTM)-based deep learning model on the CMIP6 climate dataset, effectively handling non-IID and geographically distributed data. The proposed AFL algorithm addresses key limitations of traditional FL methods, such as inefficiency due to global synchronization and susceptibility to client drift. It enhances scalability, robustness, and efficiency in real-world settings with heterogeneous client populations and dynamic network conditions. Our results underscore the potential of AFL to drive advancements in distributed learning systems, particularly for large-scale, privacy-preserving applications in resource-constrained environments.

Boosting Asynchronous Decentralized Learning with Model Fragmentation

Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees

Noiseless Privacy-Preserving Decentralized Learning

Decentralized Federated Learning under Communication Delays.

Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging

Secure Aggregation Meets Sparsification in Decentralized Learning

LSDDL: Layer-Wise Sparsification for Distributed Deep Learning

Fair Decentralized Learning

Energy-Aware Decentralized Learning with Intermittent Model Training

Communication-and-Computation Efficient Split Federated Learning: Gradient Aggregation and Resource Management

Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates

Decentralized Federated Learning: Balancing Communication and Computing Costs

Improving Model Consistency of Decentralized Federated Learning via Sharpness Aware Minimization and Multiple Gossip Approaches

Epidemic Learning: Boosting Decentralized Learning with Randomized Communication

Asynchronous Federated Learning: A Scalable Approach for Decentralized Machine Learning

Improving the Model Consistency of Decentralized Federated Learning

DFML: Decentralized Federated Mutual Learning

Accelerating Deep Learning Systems Via Critical Set Identification and Model Compression.

Decentralized Federated Learning with Unreliable Communications

DSANA: A Distributed Machine Learning Acceleration Solution Based on Dynamic Scheduling and Network Acceleration

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication