Abstract:Federated Learning (FL) provides a privacy-preserving and decentralized approach to collaborative machine learning for multiple FL clients. The contribution estimation mechanism in FL is extensively studied within the database community, which aims to compute fair and reasonable contribution scores as incentives to motivate FL clients. However, designing such methods involves challenges in three aspects: effectiveness, robustness, and efficiency. Firstly, contribution estimation methods should utilize the data utility information of various client coalitions rather than that of individual clients to ensure effectiveness. Secondly, we should beware of adverse clients who may exploit tactics like data replication or label flipping. Thirdly, estimating contribution in FL can be time-consuming due to enumerating various client coalitions. Despite numerous proposed methods to address these challenges, each possesses distinct advantages and limitations based on specific settings. However, existing methods have yet to be thoroughly evaluated and compared in the same experimental framework. Therefore, a unified and comprehensive evaluation framework is necessary to compare these methods under the same experimental settings. This paper conducts an extensive survey of contribution estimation methods in FL and introduces a comprehensive framework to evaluate their effectiveness, robustness, and efficiency. Through empirical results, we present extensive observations, valuable discoveries, and an adaptable testing framework that can facilitate future research in designing and evaluating contribution estimation methods in FL.

Fast, Robust and Interpretable Participant Contribution Estimation for Federated Learning

Contributions Estimation in Federated Learning: A Comprehensive Experimental Evaluation

Evaluate the Contribution of Multiple Participants in Federated Learning

Towards Fair, Robust and Efficient Client Contribution Evaluation in Federated Learning

A Fair Contribution Measurement Method for Federated Learning

Efficient Client Contribution Evaluation for Horizontal Federated Learning

FairFed: Improving Fairness and Efficiency of Contribution Evaluation in Federated Learning via Cooperative Shapley Value

Don't Forget What I did?: Assessing Client Contributions in Federated Learning

Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations

Redefining Contributions: Shapley-Driven Federated Learning

Intrinsic Performance Influence-based Participant Contribution Estimation for Horizontal Federated Learning.

Contribution Evaluation in Federated Learning: Examining Current Approaches

Efficient Participant Contribution Evaluation for Horizontal and Vertical Federated Learning

Data-Free Evaluation of User Contributions in Federated Learning

Measure Contribution of Participants in Federated Learning

FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning

Contribution‐based Federated Learning client selection

Contribution Matching-Based Hierarchical Incentive Mechanism Design for Crowd Federated Learning

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

CDFed: Contribution-based Dynamic Federated Learning for Managing System and Statistical Heterogeneity