Abstract:In this article, we investigate random client selection in the context of horizontal federated learning (FL), whereby only a randomly selected subset of clients transmit their model updates to the server instead of yielding all clients involved. Many researchers have demonstrated that clustering-based client selection constitutes a simple yet efficacious approach to the identification of those clients possessing representative gradient information. Despite the extensive body of research on modified selection methodologies, the majority of prior work is predicated upon the assumption of consistently effective clustering. However, raw gradient-based clustering methods are subject to several challenges: 1) poor effectiveness, the raw high-dimensional gradient of a client is too complex to serve as an appropriate feature for grouping, resulting in large intra-cluster distances and 2) fluctuating effectiveness, due to inherent limitations in clustering, the effectiveness can vary significantly, leading to clusters with diverse levels of heterogeneity. In practice, suboptimal and inconsistent clustering effects can result in clusters with low intra-cluster similarity among clients. The selection of clients from such clusters may impede the overall convergence of training. In this article, we propose, a novel client selection scheme to accelerate the FL convergence by variance reduction. The main idea of is to stratify a compressed model update in order to ensure an excellent grouping effect, and at the same time reduce the cross-client variance by re-allocating the sample chance among different groups based on their diverse heterogeneity. It strikes this convergence acceleration by paying more attention to those client groups with relatively low similarity and then improving the representativeness of the selected subset as much as possible. Theoretically, we demonstrate the critical improvement of the proposed scheme in variance reduction and present equivalence conditions among different client selection methods. We also present the tighter convergence guarantee of the proposed method thanks to the variance reduction. Experimental results confirm the exceeded efficiency of our approach compared to alternatives.

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Federated Learning With Client Selection and Gradient Compression in Heterogeneous Edge Systems

FedDGP: Disentangling Global and Personal Models for Federated Learning

FedConv: A Learning-on-Model Paradigm for Heterogeneous Federated Clients

FedSCS: Client Selection for Federated Learning under System Heterogeneity and Client Fairness with a Stackelberg Game Approach.

Federated Learning Client Selection Mechanism under System and Data Heterogeneity

FedSTS: A Stratified Client Selection Framework for Consistently Fast Federated Learning

Communication-Efficient Federated Learning With Data and Client Heterogeneity

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

FedAWARE: Maximizing Gradient Diversity for Heterogeneous Federated Server-side Optimization

Fast Heterogeneous Federated Learning with Hybrid Client Selection

FedGH: Heterogeneous Federated Learning with Generalized Global Header

FedHC: A Scalable Federated Learning Framework for Heterogeneous and Resource-Constrained Clients

Accelerating Federated Learning via Sequential Training of Grouped Heterogeneous Clients

FedAgg: Adaptive Federated Learning with Aggregated Gradients

Communication-Efficient Federated Learning with Accelerated Client Gradient

Addressing Heterogeneity in Federated Learning with Client Selection via Submodular Optimization

Federated Learning under Heterogeneous and Correlated Client Availability

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

CDFed: Contribution-based Dynamic Federated Learning for Managing System and Statistical Heterogeneity