Abstract:Federated learning is highly susceptible to model poisoning attacks, especially those meticulously crafted for servers. Traditional defense methods mainly focus on updating assessments or robust aggregation against manually crafted myopic attacks. When facing advanced attacks, their defense stability is notably insufficient. Therefore, it is imperative to develop adaptive defenses against such advanced poisoning attacks. We find that benign clients exhibit significantly higher data distribution stability than malicious clients in federated learning in both CV and NLP tasks. Therefore, the malicious clients can be recognized by observing the stability of their data distribution. In this paper, we propose AdaAggRL, an RL-based Adaptive Aggregation method, to defend against sophisticated poisoning attacks. Specifically, we first utilize distribution learning to simulate the clients' data distributions. Then, we use the maximum mean discrepancy (MMD) to calculate the pairwise similarity of the current local model data distribution, its historical data distribution, and global model data distribution. Finally, we use policy learning to adaptively determine the aggregation weights based on the above similarities. Experiments on four real-world datasets demonstrate that the proposed defense model significantly outperforms widely adopted defense models for sophisticated attacks.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to defend against the carefully - designed model poisoning attacks (Model Poisoning Attacks) against the server in federated learning (Federated Learning, FL). Specifically, traditional defense methods mainly focus on update evaluation mechanisms or use robust aggregation methods to resist short - sighted attacks manually constructed, but in the face of advanced attacks, the defense stability of these methods is obviously insufficient. Therefore, this paper aims to develop adaptive defense strategies to deal with such advanced poisoning attacks. ### Core Problems of the Paper 1. **Model Poisoning Attacks in Federated Learning**: - Federated learning systems are vulnerable to model poisoning attacks from unknown clients, especially those carefully designed against the central server. - The goal of these attacks is to minimize the accuracy of the global model by sending customized gradients to the server. 2. **Limitations of Existing Defense Methods**: - Traditional defense methods mainly rely on designing local model update evaluation mechanisms or using robust aggregation methods to mitigate the impact of poisoning attacks. - These methods have significantly insufficient defense stability in the face of advanced attacks. 3. **The Proposed New Method**: - This paper proposes an adaptive aggregation method based on reinforcement learning (AdaAggRL) to defend against complex model poisoning attacks. - This method identifies malicious clients by observing the stability of client - data distributions and adaptively determines aggregation weights according to these stability indicators. ### Solutions 1. **Data Distribution Stability**: - The authors find that benign clients show significant data distribution stability in federated learning, while the data distribution of malicious clients lacks regularity. - Malicious clients can be identified by simulating the data distribution of clients and calculating the similarity between the current data distribution and the historical data distribution and the global data distribution. 2. **Maximum Mean Discrepancy (MMD)**: - Use MMD to calculate the pairwise similarity between the current local - model data distribution and its historical data distribution and the global - model data distribution. 3. **Reinforcement Learning (RL)**: - Use policy - learning methods (such as the TD3 algorithm) to adaptively determine aggregation weights based on the above - mentioned similarities and the reconstructed similarities of distribution learning. ### Experimental Results - Experiments were carried out on four real - world datasets, and the results show that the proposed AdaAggRL defense model is significantly superior to widely - adopted defense models in the face of complex attacks. - In particular, in the face of reinforcement - learning - based attacks, AdaAggRL can maintain stable global - model accuracy. ### Innovation Points 1. **Propose an adaptive aggregation method based on reinforcement learning, AdaAggRL**, which is used to defend against complex non - target model poisoning attacks against the server and promotes the further development of defense systems. 2. **Observe the stability of data distribution of benign clients in CV and NLP tasks**, which is in sharp contrast to the irregular distribution of malicious clients. 3. **Use four indicators as RL environmental cues** and dynamically adjust the aggregation weights of local models through policy learning. ### Conclusion In this paper, by proposing the AdaAggRL method, the defense problem of advanced model poisoning attacks in federated learning is effectively solved, and the robustness and stability of the system are improved.

Defending Against Sophisticated Poisoning Attacks with RL-based Aggregation in Federated Learning

Oblivion: Poisoning Federated Learning by Inducing Catastrophic Forgetting.

Poisoning Deep Learning Based Recommender Model in Federated Learning Scenarios.

RFLPA: A Robust Federated Learning Framework against Poisoning Attacks with Secure Aggregation

RoseAgg: Robust Defense Against Targeted Collusion Attacks in Federated Learning

FedRRA: Reputation-Aware Robust Federated Learning Against Poisoning Attacks.

Defending Poisoning Attacks in Federated Learning via Adversarial Training Method

PnA: Robust Aggregation Against Poisoning Attacks to Federated Learning for Edge Intelligence

Efficient and Privacy-Preserving Federated Learning Against Poisoning Adversaries

Breaking State-of-the-Art Poisoning Defenses to Federated Learning: an Optimization-Based Attack Framework

Secure Model Aggregation Against Poisoning Attacks for Cross-Silo Federated Learning With Robustness and Fairness

ADFL: A Poisoning Attack Defense Framework for Horizontal Federated Learning

Mitigation of a poisoning attack in federated learning by using historical distance detection

Romoa: Robust Model Aggregation for the Resistance of Federated Learning to Model Poisoning Attacks

FedRMA: A Robust Federated Learning Resistant to Multiple Poisoning Attacks

Dynamic Defense Against Byzantine Poisoning Attacks in Federated Learning

Detecting and mitigating poisoning attacks in federated learning using generative adversarial networks

Poisoning Attack Detection Using Client Historical Similarity in Non-Iid Environments

Depriving the Survival Space of Adversaries Against Poisoned Gradients in Federated Learning

Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning