Abstract:In Federated Learning (FL), a set of clients collaboratively train a machine learning model (called global model) without sharing their local training data. The local training data of clients is typically non-i.i.d. and heterogeneous, resulting in varying contributions from individual clients to the final performance of the global model. In response, many contribution evaluation methods were proposed, where the server could evaluate the contribution made by each client and incentivize the high-contributing clients to sustain their long-term participation in FL. Existing studies mainly focus on developing new metrics or algorithms to better measure the contribution of each client. However, the security of contribution evaluation methods of FL operating in adversarial environments is largely unexplored. In this paper, we propose the first model poisoning attack on contribution evaluation methods in FL, termed ACE. Specifically, we show that any malicious client utilizing ACE could manipulate the parameters of its local model such that it is evaluated to have a high contribution by the server, even when its local training data is indeed of low quality. We perform both theoretical analysis and empirical evaluations of ACE. Theoretically, we show our design of ACE can effectively boost the malicious client's perceived contribution when the server employs the widely-used cosine distance metric to measure contribution. Empirically, our results show ACE effectively and efficiently deceive five state-of-the-art contribution evaluation methods. In addition, ACE preserves the accuracy of the final global models on testing inputs. We also explore six countermeasures to defend ACE. Our results show they are inadequate to thwart ACE, highlighting the urgent need for new defenses to safeguard the contribution evaluation methods in FL.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in Federated Learning (FL), how malicious clients can manipulate the contribution evaluation method through model poisoning attacks, so that their contribution scores on the server are wrongly increased. Specifically, the paper proposes a model poisoning attack named ACE (Attack on Contribution Evaluation). This attack allows malicious clients to manipulate their local model parameters even if they have low - quality local training data, making the server evaluate their contribution as high. This may not only lead to malicious clients obtaining additional rewards, but also damage the interests of other participants and undermine the fairness of the Federated Learning system.
### Main contributions of the paper:
1. **Proposing the ACE attack**: This is the first model poisoning attack targeting the contribution evaluation method in Federated Learning.
2. **Theoretical analysis and empirical evaluation**: The paper conducts a detailed theoretical analysis of ACE and demonstrates its effectiveness and efficiency through extensive empirical evaluation.
3. **Exploration of defense measures**: The paper explores six strategies to defend against ACE and finds that these strategies are largely unable to effectively defend against ACE, emphasizing the urgency of developing new defense mechanisms.
### Background and related work:
- **Federated Learning (FL)**: Multiple clients collaborate to train a global model without sharing local data. Each client uses local data to update its local model and sends the update to the server, and the server aggregates these updates to update the global model.
- **Contribution evaluation methods**: In order to encourage holders of high - quality data to actively participate in Federated Learning, researchers have proposed a variety of contribution evaluation methods, including self - reporting - based methods, individual - performance - based methods, and game - theory - based methods.
### Problem modeling:
- **Threat model**: The attacker has multiple malicious clients, can access the local data and global model of these clients, control the training process of the local model, and manipulate its parameters before sending the local model update. The attacker's goal is to increase the contribution score of the malicious clients on the server.
- **Optimization problem**: The attacker needs to design a method to maximize the cumulative contribution of the malicious clients. Formally, this problem can be expressed as the following optimization problem:
\[
\{ \hat{g}_t^i | i \in \hat{\Gamma} \} = \arg\max_{\{ g_i | i \in \hat{\Gamma} \}} \sum_{i \in \hat{\Gamma}} E(g_i)
\]
where \(\hat{\Gamma}\) is the set of malicious clients, \(g_i\) is the local model update of the \(i\)-th client, and \(E(g_i)\) is the contribution value calculated by the server.
### Design of the ACE attack:
- **Future global model prediction**: Malicious clients adjust their local model updates by predicting the future global model to make them closer to the predicted global model, thereby increasing the contribution score.
- **Prediction error mitigation**: Since prediction errors may accumulate, the paper proposes two strategies to mitigate this problem: preliminary iteration and threshold filtering.
- **Preliminary iteration**: In the initial communication rounds, malicious clients use local data or the global model update of the previous round as a proxy to collect historical information to build a buffer.
- **Threshold filtering**: Set a threshold to estimate whether the prediction error is too large. If the error is too large, use local data or the global model update of the previous round.
### Adaptation to specific contribution evaluation methods:
- **Cosine distance method**: When the server uses the cosine distance to evaluate the contribution, the optimization problem can be rewritten as:
\[
\{ \hat{g}_t^i | i \in \hat{\Gamma} \} = \arg\max_{\{ g_i | i \in \hat{\Gamma} \}} \sum_{i \in \hat{\Gamma}} (1 - \cos(g_i, g_t))
\]
where \(\cos(a, b) = 1 - \