Abstract:This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which, in turn, aggregates them into a quantized global model and synchronizes the devices. The goal is to jointly determine the bitwidths employed for local FL model quantization and the set of devices participating in FL training at each iteration. We pose this as an optimization problem that aims to minimize the training loss of quantized FL under a per-iteration device sampling budget and delay requirement. However, the formulated problem is difficult to solve without (i) a concrete understanding of how quantization impacts global ML performance and (ii) the ability of the server to construct estimates of this process efficiently. To address the first challenge, we analytically characterize how limited wireless resources and induced quantization errors affect the performance of the proposed FL method. Our results quantify how the improvement of FL training loss between two consecutive iterations depends on the device selection and quantization scheme as well as on several parameters inherent to the model being learned. Then, to address the second challenge, we show that the FL training process can be described as a Markov decision process (MDP) and propose a model-based reinforcement learning (RL) method to optimize action selection over iterations. Compared to model-free RL, this model-based RL approach leverages the derived mathematical characterization of the FL training process to discover an effective device selection and quantization scheme without imposing additional device communication overhead. Simulation results show that the proposed FL algorithm can reduce the convergence time by 29% and 63% compared to a model free RL method and the standard FL method, respectively.

A Communication-Efficient Federated Learning by Dynamic Quantization and Free-Ride Coding

Deep Learning Based Coded Over-the-Air Computation for Personalized Federated Learning

Federated Optimal Framework with Low-bitwidth Quantization for Distribution System

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization

FedDQ: Communication-Efficient Federated Learning with Descending Quantization

Design and Analysis of Uplink and Downlink Communications for Federated Learning

FedFQ: Federated Learning with Fine-Grained Quantization

Communication-Efficient Design for Quantized Decentralized Federated Learning

Quantized Federated Learning under Transmission Delay and Outage Constraints

Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

Quantization Bits Allocation for Wireless Federated Learning

Communication Efficient Federated Learning with Adaptive Quantization

NQFL: Nonuniform Quantization for Communication Efficient Federated Learning

FedMDC: Enabling Communication-Efficient Federated Learning over Packet Lossy Networks Via Multiple Description Coding

Communication-Efficient Federated Learning with Sparsity and Quantization

Towards Efficient Federated Learning: Layer-Wise Pruning-Quantization Scheme and Coding Design.

Energy-Efficient Wireless Federated Learning via Doubly Adaptive Quantization

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

FedVQCS: Federated Learning via Vector Quantized Compressed Sensing