Abstract:Federated Learning (FL) provides a privacy-preserving and decentralized approach to collaborative machine learning for multiple FL clients. The contribution estimation mechanism in FL is extensively studied within the database community, which aims to compute fair and reasonable contribution scores as incentives to motivate FL clients. However, designing such methods involves challenges in three aspects: effectiveness, robustness, and efficiency. Firstly, contribution estimation methods should utilize the data utility information of various client coalitions rather than that of individual clients to ensure effectiveness. Secondly, we should beware of adverse clients who may exploit tactics like data replication or label flipping. Thirdly, estimating contribution in FL can be time-consuming due to enumerating various client coalitions. Despite numerous proposed methods to address these challenges, each possesses distinct advantages and limitations based on specific settings. However, existing methods have yet to be thoroughly evaluated and compared in the same experimental framework. Therefore, a unified and comprehensive evaluation framework is necessary to compare these methods under the same experimental settings. This paper conducts an extensive survey of contribution estimation methods in FL and introduces a comprehensive framework to evaluate their effectiveness, robustness, and efficiency. Through empirical results, we present extensive observations, valuable discoveries, and an adaptable testing framework that can facilitate future research in designing and evaluating contribution estimation methods in FL.

Fair and efficient contribution valuation for vertical federated learning

Evaluate the Contribution of Multiple Participants in Federated Learning

A Principled Approach to Data Valuation for Federated Learning

Improving Fairness for Data Valuation in Horizontal Federated Learning

FairFed: Improving Fairness and Efficiency of Contribution Evaluation in Federated Learning via Cooperative Shapley Value

Data Valuation for Vertical Federated Learning: An Information-Theoretic Approach

A Fair Contribution Measurement Method for Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Data Valuation for Vertical Federated Learning: A Model-free and Privacy-preserving Method

Shapley-value-based Contribution Evaluation in Federated Learning: A Survey

A Survey on Contribution Evaluation in Vertical Federated Learning

PoShapley-BCFL: A Fair and Robust Decentralized Federated Learning Based on Blockchain and the Proof of Shapley-Value

Efficient Participant Contribution Evaluation for Horizontal and Vertical Federated Learning

DPVS-Shapley:Faster and Universal Contribution Evaluation Component in Federated Learning

Redefining Contributions: Shapley-Driven Federated Learning

Efficient Client Contribution Evaluation for Horizontal Federated Learning

Achieving Model Fairness in Vertical Federated Learning

Maverick-Aware Shapley Valuation for Client Selection in Federated Learning

Secure Shapley Value for Cross-Silo Federated Learning (Technical Report)

Contributions Estimation in Federated Learning: A Comprehensive Experimental Evaluation

FedSV: Byzantine-Robust Federated Learning via Shapley Value