Greedy Shapley Client Selection for Communication-Efficient Federated Learning

Pranava Singhal,Shashi Raj Pandey,Petar Popovski

DOI: https://doi.org/10.1109/LNET.2024.3363620

2024-02-07

Abstract:The standard client selection algorithms for Federated Learning (FL) are often unbiased and involve uniform random sampling of clients. This has been proven sub-optimal for fast convergence under practical settings characterized by significant heterogeneity in data distribution, computing, and communication resources across clients. For applications having timing constraints due to limited communication opportunities with the parameter server (PS), the client selection strategy is critical to complete model training within the fixed budget of communication rounds. To address this, we develop a biased client selection strategy, GreedyFed, that identifies and greedily selects the most contributing clients in each communication round. This method builds on a fast approximation algorithm for the Shapley Value at the PS, making the computation tractable for real-world applications with many clients. Compared to various client selection strategies on several real-world datasets, GreedyFed demonstrates fast and stable convergence with high accuracy under timing constraints and when imposing a higher degree of heterogeneity in data distribution, systems constraints, and privacy requirements.

Machine Learning,Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

The main problem addressed by this paper is how to improve communication efficiency and model convergence speed in Federated Learning (FL) by enhancing client selection strategies. Specifically, the paper proposes a new method called **GREEDY FED**, which selects clients that contribute the most to training based on Shapley values. This method has several advantages over traditional random selection or uniform sampling strategies in the following aspects: 1. **Data Heterogeneity**: In cases where client data distribution is highly uneven, GREEDY FED can better identify clients that contribute significantly to the global model, thereby accelerating the model convergence process. 2. **Time Constraints**: When there are limitations on communication rounds, GREEDY FED can achieve higher accuracy within fewer communication rounds, which is particularly important for real-time applications in practical deployments. 3. **System Heterogeneity**: GREEDY FED demonstrates good robustness in the face of differences in computing power and communication resources among clients. 4. **Privacy Heterogeneity**: When clients need to protect their data privacy by adding noise, GREEDY FED shows stronger stability in handling noisy data. The paper validates the effectiveness of the GREEDY FED method through experiments on multiple real-world datasets and compares it with various existing methods, demonstrating its consistently superior performance under various conditions.

Greedy Shapley Client Selection for Communication-Efficient Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Bandit-based Communication-Efficient Client Selection Strategies for Federated Learning

Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Game of Gradients: Mitigating Irrelevant Clients in Federated Learning

Communication-Efficient Federated Learning with Accelerated Client Gradient

FedSTS: A Stratified Client Selection Framework for Consistently Fast Federated Learning

FedSS: Federated Learning with Smart Selection of clients

Fairness-Aware Client Selection for Federated Learning

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

DPP-based Client Selection for Federated Learning with Non-IID Data

FLAS: Computation and Communication Efficient Federated Learning via Adaptive Sampling

Tackling Mavericks in Federated Learning via Adaptive Client Selection Strategy

Communication-Efficient Federated Learning With Data and Client Heterogeneity

Is Shapley Value fair? Improving Client Selection for Mavericks in Federated Learning

Federated Dynamic Client Selection for Fairness Guarantee in Heterogeneous Edge Computing

Client Selection in Federated Learning: Principles, Challenges, and Opportunities

FedAHP: A Heterogeneous Client Selection Method for Federated Learning Based on the Analytic Hierarchy Process in Mobile Edge

ShapleyFL: Robust Federated Learning Based on Shapley Value