Abstract:Originated from distributed learning, federated learning enables privacy-preserved collaboration on a new abstracted level by sharing the model parameters only. While the current research mainly focuses on optimizing learning algorithms and minimizing communication overhead left by distributed learning, there is still a considerable gap when it comes to the real implementation on mobile devices. In this article, we start with an empirical experiment to demonstrate computation heterogeneity is a more pronounced bottleneck than communication on the current generation of battery-powered mobile devices, and the existing methods are haunted by mobile stragglers. Further, non-identically distributed data across the mobile users makes the selection of participants critical to the accuracy and convergence. To tackle the computational and statistical heterogeneity, we utilize data as a tuning knob and propose two efficient polynomial-time algorithms to schedule different workloads on various mobile devices, when data is identically or non-identically distributed. For identically distributed data, we combine partitioning and linear bottleneck assignment to achieve near-optimal training time without accuracy loss. For non-identically distributed data, we convert it into an average cost minimization problem and propose a greedy algorithm to find a reasonable balance between computation time and accuracy. We also establish an offline profiler to quantify the runtime behavior of different devices, which serves as the input to the scheduling algorithms. We conduct extensive experiments on a mobile testbed with two datasets and up to 20 devices. Compared with the common benchmarks, the proposed algorithms achieve 2-100× speedup epoch-wise, 2–7 percent accuracy gain and boost the convergence rate by more than 100 percent on CIFAR10.

Efficient Scheduling for Multi-Job Federated Learning Systems with Client Sharing

Towards Efficient Scheduling of Federated Mobile Devices under Computational and Statistical Heterogeneity

Efficient Device Scheduling with Multi-Job Federated Learning

Fairness-Aware Job Scheduling for Multi-Job Federated Learning

Multi-Job Intelligent Scheduling with Cross-Device Federated Learning

Joint Client Scheduling and Wireless Resource Allocation for Heterogeneous Federated Edge Learning with Non-IID Data

Federated learning with workload-aware client scheduling in heterogeneous systems

Efficient Scheduling for Multi-Job Vertical Federated Learning

Online Client Scheduling for Fast Federated Learning

Resource Allocation for Multi-Task Federated Learning Algorithm over Wireless Communication Networks

FedSCS: Client Selection for Federated Learning under System Heterogeneity and Client Fairness with a Stackelberg Game Approach.

Power of Redundancy: Surplus Client Scheduling for Federated Learning Against User Uncertainties

FedRich: Towards Efficient Federated Learning for Heterogeneous Clients Using Heuristic Scheduling

Multi-Criteria Client Selection and Scheduling with Fairness Guarantee for Federated Learning Service

Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Heterogeneous Federated Learning for Balancing Job Completion Time and Model Accuracy

Client Grouping and Time-Sharing Scheduling for Asynchronous Federated Learning in Heterogeneous Edge Computing Environment

Mobility-Aware Joint User Scheduling and Resource Allocation for Low Latency Federated Learning

Fair Concurrent Training of Multiple Models in Federated Learning