Abstract:Federated learning is a new distributed machine learning framework, where numerous heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue when deploying federated learning in mobile environments: intermittent client availability , where the set of eligible clients may change during the training process. Such intermittent client availability would seriously deteriorate the performance of the classical federated averaging algorithm (FedAvg). Thus, we propose a simple distributed nonconvex optimization algorithm, called federated latest averaging (FedLaAvg), which leverages the latest gradients of all clients, even when the clients are not available, to jointly update the global model in each iteration. Our theoretical analysis shows that FedLaAvg achieves guaranteed convergence and a sublinear speedup with respect to the total number of clients. We implement FedLaAvg along with several baselines and evaluate them over the benchmarking MNIST and Sentiment140 data sets. The evaluation results demonstrate that FedLaAvg achieves more stable training than FedAvg in both convex and nonconvex settings and reaches a sublinear speedup. Source code and online supplement are available at the IJOC GitHub site ( http://dx.doi.org/10.1287/ijoc.2022.0057.cd , https://github.com/INFORMSJoC/2022.0057 ). History: Accepted by Ram Ramesh, Area Editor for Data Science & Machine Leaning. Funding: This work was supported by the National Key R&D Program of China [Grant 2022ZD0119100], the National Natural Science Foundation of China (NSFC) [Grants 61972252, 61972254, 62072303, 62025204, 62132018, 62202296, and 62202297], the Alibaba Innovation Research (AIR) Program, and the Tencent Rhino Bird Key Research Project. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0057 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0057 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Energy Optimization for Federated Learning on Consumer Mobile Devices with Asynchronous SGD and Application Co-Execution

Optimize Scheduling of Federated Learning on Battery-powered Mobile Devices

Towards Efficient Scheduling of Federated Mobile Devices under Computational and Statistical Heterogeneity

Energy Minimization for Federated Asynchronous Learning on Battery-Powered Mobile Devices via Application Co-running

Adaptive Batchsize Selection and Gradient Compression for Wireless Federated Learning

Energy-efficient Federated Learning Via Stabilization-aware On-device Update Scaling.

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning

Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability

Software/Code for "federated Optimization under Intermittent Client Availability"

Device Scheduling and Resource Allocation for Federated Learning under Delay and Energy Constraints

Federated Optimization under Intermittent Client Availability

Gradient and Channel Aware Dynamic Scheduling for Over-the-Air Computation in Federated Edge Learning Systems

Energy-Efficient Federated Edge Learning with Streaming Data: A Lyapunov Optimization Approach

DEAL: Decremental Energy-Aware Learning in a Federated System

Data and Channel-Adaptive Sensor Scheduling for Federated Edge Learning via Over-the-Air Gradient Aggregation

Adaptive Decentralized Federated Learning in Energy and Latency Constrained Wireless Networks

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Exploring Age-of-Information Weighting in Federated Learning under Data Heterogeneity

Energy-Aware Analog Aggregation for Federated Learning with Redundant Data

Multi-Resource Allocation for On-Device Distributed Federated Learning Systems