Abstract:Federated learning is a new distributed machine learning framework, where numerous heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue when deploying federated learning in mobile environments: intermittent client availability , where the set of eligible clients may change during the training process. Such intermittent client availability would seriously deteriorate the performance of the classical federated averaging algorithm (FedAvg). Thus, we propose a simple distributed nonconvex optimization algorithm, called federated latest averaging (FedLaAvg), which leverages the latest gradients of all clients, even when the clients are not available, to jointly update the global model in each iteration. Our theoretical analysis shows that FedLaAvg achieves guaranteed convergence and a sublinear speedup with respect to the total number of clients. We implement FedLaAvg along with several baselines and evaluate them over the benchmarking MNIST and Sentiment140 data sets. The evaluation results demonstrate that FedLaAvg achieves more stable training than FedAvg in both convex and nonconvex settings and reaches a sublinear speedup. Source code and online supplement are available at the IJOC GitHub site ( http://dx.doi.org/10.1287/ijoc.2022.0057.cd , https://github.com/INFORMSJoC/2022.0057 ). History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Leaning. Funding: This work was supported by the National Key R&D Program of China [Grant 2022ZD0119100], the National Natural Science Foundation of China (NSFC) [Grants 61972252, 61972254, 62072303, 62025204, 62132018, 62202296, and 62202297], the Alibaba Innovation Research (AIR) Program, and the Tencent Rhino Bird Key Research Project. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0057 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0057 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Optimizing Data Distributions Based on Jensen-Shannon Divergence for Federated Learning

FedDGP: Disentangling Global and Personal Models for Federated Learning

Understanding the Training Dynamics in Federated Deep Learning via Aggregation Weight Optimization

Fairness and Accuracy in Federated Learning

FedGA: A Greedy Approach to Enhance Federated Learning with Non-IID Data

FedUB: Federated Learning Algorithm Based on Update Bias

Federated Learning Optimization Algorithm for Automatic Weight Optimal

FedDistill: Global Model Distillation for Local Model De-Biasing in Non-IID Federated Learning

Optimization Strategies for Client Drift in Federated Learning: A review

FedCos: A Scene-adaptive Federated Optimization Enhancement for Performance Improvement

FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

Federated Optimization Via Knowledge Codistillation.

Federated Optimization under Intermittent Client Availability

Fed-DR-Filter: Using Global Data Representation to Reduce the Impact of Noisy Labels on the Performance of Federated Learning

FedDBO: A Novel Federated Learning Approach for Communication Cost and Data Heterogeneity Using Dung Beetle Optimizer

Data Quality-Aware Client Selection in Heterogeneous Federated Learning

FED-3DA: A Dynamic and Personalized Federated Learning Framework

DeFed: A Principled Decentralized and Privacy-Preserving Federated Learning Algorithm.

Fine-tuning Global Model Via Data-Free Knowledge Distillation for Non-IID Federated Learning

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

Federated Learning on Non-Independent and Identically Distributed Data