Abstract:Originated from distributed learning, federated learning enables privacy-preserved collaboration on a new abstracted level by sharing the model parameters only. While the current research mainly focuses on optimizing learning algorithms and minimizing communication overhead left by distributed learning, there is still a considerable gap when it comes to the real implementation on mobile devices. In this article, we start with an empirical experiment to demonstrate computation heterogeneity is a more pronounced bottleneck than communication on the current generation of battery-powered mobile devices, and the existing methods are haunted by mobile stragglers. Further, non-identically distributed data across the mobile users makes the selection of participants critical to the accuracy and convergence. To tackle the computational and statistical heterogeneity, we utilize data as a tuning knob and propose two efficient polynomial-time algorithms to schedule different workloads on various mobile devices, when data is identically or non-identically distributed. For identically distributed data, we combine partitioning and linear bottleneck assignment to achieve near-optimal training time without accuracy loss. For non-identically distributed data, we convert it into an average cost minimization problem and propose a greedy algorithm to find a reasonable balance between computation time and accuracy. We also establish an offline profiler to quantify the runtime behavior of different devices, which serves as the input to the scheduling algorithms. We conduct extensive experiments on a mobile testbed with two datasets and up to 20 devices. Compared with the common benchmarks, the proposed algorithms achieve 2-100× speedup epoch-wise, 2–7 percent accuracy gain and boost the convergence rate by more than 100 percent on CIFAR10.

TDMA-based User Scheduling Policies for Federated Learning

Towards Efficient Scheduling of Federated Mobile Devices under Computational and Statistical Heterogeneity

Fast Convergence for Federated Learning in OFDMA Systems

Resource Allocation for Multi-Task Federated Learning Algorithm over Wireless Communication Networks

User Scheduling for Federated Learning Through Over-the-Air Computation

Joint User Scheduling and Resource Allocation for Federated Learning over Wireless Networks

Device Scheduling with Fast Convergence for Wireless Federated Learning

Scheduling Policies for Federated Learning in Wireless Networks

Scheduling Policies for Federated Learning in Wireless Networks:An Overview

Mobility-Aware Joint User Scheduling and Resource Allocation for Low Latency Federated Learning

Federated Learning Over Wireless Channels: Dynamic Resource Allocation and Task Scheduling

Device Scheduling and Resource Allocation for Federated Learning under Delay and Energy Constraints

Accelerating Federated Edge Learning via Optimized Probabilistic Device Scheduling

Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Communication-efficient Federated Edge Learning via Optimal Probabilistic Device Scheduling

Time-triggered Federated Learning over Wireless Networks

To Talk or to Work: Dynamic Batch Sizes Assisted Time Efficient Federated Learning Over Future Mobile Edge Devices

Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning

Efficient Device Scheduling with Multi-Job Federated Learning

Value of Information and Timing-aware Scheduling for Federated Learning