Abstract:This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which, in turn, aggregates them into a quantized global model and synchronizes the devices. The goal is to jointly determine the bitwidths employed for local FL model quantization and the set of devices participating in FL training at each iteration. We pose this as an optimization problem that aims to minimize the training loss of quantized FL under a per-iteration device sampling budget and delay requirement. However, the formulated problem is difficult to solve without (i) a concrete understanding of how quantization impacts global ML performance and (ii) the ability of the server to construct estimates of this process efficiently. To address the first challenge, we analytically characterize how limited wireless resources and induced quantization errors affect the performance of the proposed FL method. Our results quantify how the improvement of FL training loss between two consecutive iterations depends on the device selection and quantization scheme as well as on several parameters inherent to the model being learned. Then, to address the second challenge, we show that the FL training process can be described as a Markov decision process (MDP) and propose a model-based reinforcement learning (RL) method to optimize action selection over iterations. Compared to model-free RL, this model-based RL approach leverages the derived mathematical characterization of the FL training process to discover an effective device selection and quantization scheme without imposing additional device communication overhead. Simulation results show that the proposed FL algorithm can reduce the convergence time by 29% and 63% compared to a model free RL method and the standard FL method, respectively.

Joint Sparsification and Quantization for Heterogeneous Devices in Energy Constrained Federated Learning

Latency-Efficient Wireless Federated Learning with Spasification and Quantization for Heterogeneous Devices

Joint Sparsification and Quantization for Wireless Federated Learning under Communication Constraints

Secure Federated Learning with Model Compression.

Secure Federated Learning over Wireless Communication Networks with Model Compression

Joint Optimization for Federated Learning over the Air

Communication Efficient Federated Learning Based on Combination of Structural Sparsification and Hybrid Quantization Sensing

Joint Optimization of Communications and Federated Learning over the Air

Energy Efficient Federated Learning over Heterogeneous Mobile Devices via Joint Design of Weight Quantization and Wireless Transmission

Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization

Communication-Efficient Federated Learning with Sparsity and Quantization

Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices

Quantization Bits Allocation for Wireless Federated Learning

Efficient asynchronous federated learning with sparsification and quantization

Energy-Efficient Wireless Federated Learning via Doubly Adaptive Quantization

Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design

Efficient Model Compression via Global Sparsification for Over-the-Air Federated Learning.

Joint Compression and Deadline Optimization for Wireless Federated Learning

IoT Device Friendly and Communication Efficient Federated Learning via Joint Model Pruning and Quantization

Joint Gradient Sparsification and Device Scheduling for Federated Learning