Abstract:Federated Learning (FL) with quantization and deliberately added noise over wireless networks is a promising approach to preserve user differential privacy (DP) while reducing wireless resources. Specifically, an FL process can be fused with quantized Binomial mechanism-based updates contributed by multiple users. However, optimizing quantization parameters, communication resources (e.g., transmit power, bandwidth, and quantization bits), and the added noise to guarantee the DP requirement and performance of the learned FL model remains an open and challenging problem. This article aims to jointly optimize the quantization and Binomial mechanism parameters and communication resources to maximize the convergence rate under the constraints of the wireless network and DP requirement. To that end, we first derive a novel DP budget estimation of the FL with quantization/noise that is tighter than the state-of-the-art bound. We then provide a theoretical bound on the convergence rate. This theoretical bound is decomposed into two components, including the variance of the global gradient and the quadratic bias that can be minimized by optimizing the communication resources, and quantization/noise parameters. The resulting optimization turns out to be a Mixed-Integer Non-linear Programming (MINLP) problem. To tackle it, we first transform this MINLP problem into a new problem whose solutions are proved to be the optimal solutions of the original one. We then propose an approximate algorithm to solve the transformed problem with an arbitrary relative error guarantee. Extensive simulations show that under the same wireless resource constraints and DP protection requirements, the proposed approximate algorithm achieves an accuracy close to the accuracy of the conventional FL without quantization/noise. The results can achieve a higher convergence rate while preserving users' privacy.

Towards Energy-efficient Federated Learning Via INT8-based Training on Mobile DSPs

Optimize Scheduling of Federated Learning on Battery-powered Mobile Devices

Energy Efficient Federated Learning over Heterogeneous Mobile Devices via Joint Design of Weight Quantization and Wireless Transmission

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices

Decentralized Federated Learning with Intermediate Results in Mobile Edge Computing

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization

Energy and Spectrum Efficient Federated Learning via High-Precision Over-the-Air Computation

Energy Efficient and Differentially Private Federated Learning via a Piggyback Approach

On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks

Towards Energy Efficient Federated Learning over 5G+ Mobile Devices

Adaptive Decentralized Federated Learning in Energy and Latency Constrained Wireless Networks

Efficient asynchronous federated learning with sparsification and quantization

Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point

Toward Energy-Efficient Federated Learning Over 5G+ Mobile Devices

Accelerating Federated Learning for IoT in Big Data Analytics With Pruning, Quantization and Selective Updating

Energy-Efficient Wireless Federated Learning via Doubly Adaptive Quantization

An Efficient Asynchronous Federated Learning Protocol for Edge Devices

Optimal Privacy Preserving for Federated Learning in Mobile Edge Computing

Design and Analysis of Uplink and Downlink Communications for Federated Learning

ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity