Abstract:Federated Learning (FL) enables collaborative learning of large-scale distributed clients without data sharing. However, due to the disparity of computing resources among massive mobile computing devices, the performance of traditional homogeneous model-based Federated Learning (FL) is seriously limited. On the one hand, to achieve model training in all the diverse clients, mobile computing systems can only use small low-performance models for collaborative learning. On the other hand, devices with high computing resources cannot train a high-performance large model with their insufficient raw data. To address the resource-constrained problem in mobile computing systems, we present a novel heterogeneous FL approach named AdapterFL, which uses a model reassemble strategy to facilitate collaborative training of massive heterogeneous mobile devices adaptively. Specifically, we select multiple candidate heterogeneous models based on the computing performance of massive mobile devices and then divide each heterogeneous model into two partitions. By reassembling the partitions, we can generate models with varied sizes that are combined by the partial parameters of the large model with the partial parameters of the small model. Using these reassembled models for FL training, we can train the partial parameters of the large model using low-performance devices. In this way, we can alleviate performance degradation in large models due to resource constraints. The experimental results show that AdapterFL can achieve up to 12\% accuracy improvement compared to the state-of-the-art heterogeneous federated learning methods in resource-constrained scenarios.

Towards Mitigating Device Heterogeneity in Federated Learning via Adaptive Model Quantization

Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices

AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy

Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices

Communication Efficient Federated Learning with Adaptive Quantization

Ensemble Distillation based Adaptive Quantization for Supporting Federated Learning in Wireless Networks

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization

Joint Sparsification and Quantization for Heterogeneous Devices in Energy Constrained Federated Learning

FedFQ: Federated Learning with Fine-Grained Quantization

QuAsyncFL: Asynchronous Federated Learning With Quantization for Cloud–Edge–Terminal Collaboration Enabled AIoT

Efficient asynchronous federated learning with sparsification and quantization

QuAsyncFL: Asynchronous Federated Learning with Quantization for CloudEdgeTerminal Collaboration Enabled AIoT

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation

Latency-Efficient Wireless Federated Learning with Spasification and Quantization for Heterogeneous Devices

Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design

Adaptive Quantization Resolution and Power Control for Federated Learning over Cell-free Networks

Federated Learning for Computationally Constrained Heterogeneous Devices: A Survey

Device Heterogeneity in Federated Learning: A Superquantile Approach

AdaptFL: Adaptive Federated Learning Framework for Heterogeneous Devices

AdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systems

Accelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantization