Abstract:Federated learning (FL) has recently gained tremendous attention in edge computing and Internet of Things, due to its capability of enabling distributed clients to cooperatively train models while keeping raw data locally. However, the existing works usually suffer from limited communication resources, dynamic network conditions and heterogeneous client properties, which hinder efficient FL. To simultaneously tackle the above challenges, we propose a heterogeneity-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Specifically, FedCG introduces diversity to client selection and aims to select a representative client subset considering statistical heterogeneity. These selected clients are assigned different compression ratios based on heterogeneous and time-varying capabilities. After local training, they upload sparse model updates matching their capabilities for global aggregation, which can effectively reduce the communication cost and mitigate the straggler effect. More importantly, instead of naively combining client selection and gradient compression, we highlight that their decisions are tightly coupled and indicate the necessity of joint optimization. We theoretically analyze the impact of both client selection and gradient compression on convergence performance. Guided by the convergence rate, we develop an iteration-based algorithm to jointly optimize client selection and compression ratio decision using submodular maximization and linear programming. On this basis, we propose the quantized extension of FedCG, termed Q-FedCG, which further adjusts quantization levels based on gradient innovation. Extensive experiments on both real-world prototypes and simulations show that FedCG and its extension can provide up to 6.4× speedup.

FedQClip: Accelerating Federated Learning Via Quantized Clipped SGD

Communication-efficient Federated Learning Via Quantized Clipped SGD

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Understanding the Training Dynamics in Federated Deep Learning via Aggregation Weight Optimization

FedFQ: Federated Learning with Fine-Grained Quantization

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

Federated Learning With Client Selection and Gradient Compression in Heterogeneous Edge Systems

Efficient asynchronous federated learning with sparsification and quantization

Hierarchical Quantized Federated Learning: Convergence Analysis and System Design

Faster Federated Learning with Decaying Number of Local SGD Steps

FedAgg: Adaptive Federated Learning with Aggregated Gradients

Optimizing Quantum Federated Learning Based on Federated Quantum Natural Gradient Descent

Communication Efficient Federated Learning with Adaptive Quantization

Decentralized Federated Learning: Balancing Communication and Computing Costs

Two-layer accumulated quantized compression for communication-efficient federated learning: TLAQC

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

Preconditioned Federated Learning

FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning

FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models

Rate distortion optimization for adaptive gradient quantization in federated learning

Stochastic gradient compression for federated learning over wireless network