Abstract:Federated Learning (FL) provides a privacy-preserving and decentralized approach to collaborative machine learning for multiple FL clients. The contribution estimation mechanism in FL is extensively studied within the database community, which aims to compute fair and reasonable contribution scores as incentives to motivate FL clients. However, designing such methods involves challenges in three aspects: effectiveness, robustness, and efficiency. Firstly, contribution estimation methods should utilize the data utility information of various client coalitions rather than that of individual clients to ensure effectiveness. Secondly, we should beware of adverse clients who may exploit tactics like data replication or label flipping. Thirdly, estimating contribution in FL can be time-consuming due to enumerating various client coalitions. Despite numerous proposed methods to address these challenges, each possesses distinct advantages and limitations based on specific settings. However, existing methods have yet to be thoroughly evaluated and compared in the same experimental framework. Therefore, a unified and comprehensive evaluation framework is necessary to compare these methods under the same experimental settings. This paper conducts an extensive survey of contribution estimation methods in FL and introduces a comprehensive framework to evaluate their effectiveness, robustness, and efficiency. Through empirical results, we present extensive observations, valuable discoveries, and an adaptable testing framework that can facilitate future research in designing and evaluating contribution estimation methods in FL.

Measure Contribution of Participants in Federated Learning

Evaluate the Contribution of Multiple Participants in Federated Learning

A Fair Contribution Measurement Method for Federated Learning

Efficient Client Contribution Evaluation for Horizontal Federated Learning

Contributions Estimation in Federated Learning: A Comprehensive Experimental Evaluation

Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations

Efficient Participant Contribution Evaluation for Horizontal and Vertical Federated Learning

Fast, Robust and Interpretable Participant Contribution Estimation for Federated Learning

Don't Forget What I did?: Assessing Client Contributions in Federated Learning

Contribution Matching-Based Hierarchical Incentive Mechanism Design for Crowd Federated Learning

Redefining Contributions: Shapley-Driven Federated Learning

FairFed: Improving Fairness and Efficiency of Contribution Evaluation in Federated Learning via Cooperative Shapley Value

Fair and efficient contribution valuation for vertical federated learning

A Survey on Contribution Evaluation in Vertical Federated Learning

An Exploratory Analysis on Users' Contributions in Federated Learning

Intrinsic Performance Influence-based Participant Contribution Estimation for Horizontal Federated Learning.

DPVS-Shapley:Faster and Universal Contribution Evaluation Component in Federated Learning

Contribution Evaluation in Federated Learning: Examining Current Approaches

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Towards Fair, Robust and Efficient Client Contribution Evaluation in Federated Learning

Data-Free Evaluation of User Contributions in Federated Learning