Abstract:Federated learning (FL) has recently gained tremendous attention in edge computing and Internet of Things, due to its capability of enabling distributed clients to cooperatively train models while keeping raw data locally. However, the existing works usually suffer from limited communication resources, dynamic network conditions and heterogeneous client properties, which hinder efficient FL. To simultaneously tackle the above challenges, we propose a heterogeneity-aware FL framework, called FedCG, with adaptive client selection and gradient compression. Specifically, FedCG introduces diversity to client selection and aims to select a representative client subset considering statistical heterogeneity. These selected clients are assigned different compression ratios based on heterogeneous and time-varying capabilities. After local training, they upload sparse model updates matching their capabilities for global aggregation, which can effectively reduce the communication cost and mitigate the straggler effect. More importantly, instead of naively combining client selection and gradient compression, we highlight that their decisions are tightly coupled and indicate the necessity of joint optimization. We theoretically analyze the impact of both client selection and gradient compression on convergence performance. Guided by the convergence rate, we develop an iteration-based algorithm to jointly optimize client selection and compression ratio decision using submodular maximization and linear programming. On this basis, we propose the quantized extension of FedCG, termed Q-FedCG, which further adjusts quantization levels based on gradient innovation. Extensive experiments on both real-world prototypes and simulations show that FedCG and its extension can provide up to 6.4× speedup.

Gradient Free Personalized Federated Learning

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

PGFed: Personalize Each Client's Global Objective for Federated Learning

Communication-Efficient Zeroth-Order Adaptive Optimization for Federated Learning

Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradients

Decentralized Directed Collaboration for Personalized Federated Learning

Byzantine-resilient Federated Learning Employing Normalized Gradients on Non-IID Datasets

Efficient Personalized Federated Learning via Sparse Model-Adaptation

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

FedFTHA: A Fine-Tuning and Head Aggregation Method in Federated Learning

FedGH: Heterogeneous Federated Learning with Generalized Global Header

Depersonalized Federated Learning: Tackling Statistical Heterogeneity by Alternating Stochastic Gradient Descent

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Preconditioned Federated Learning

Sparse Federated Learning with Hierarchical Personalized Models

Group Personalized Federated Learning

Federated Learning With Client Selection and Gradient Compression in Heterogeneous Edge Systems

Sparse Personalized Federated Learning

ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning

FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent