Abstract:Federated Learning (FL) is nowadays one of the most promising paradigms for privacy-preserving distributed learning. Without revealing its local private data to outsiders, a client in FL systems collaborates to build a global Deep Neural Network (DNN) by submitting its local model parameter update to a central server for iterative aggregation. With secure multi-party computation protocols, the submitted update of any client is also by design invisible to the server. Seemingly, this standard design is a win-win for client privacy and service provider utility. Ironically, any attacker may also use manipulated or impersonated client to submit almost any attack payloads under the umbrella of the FL protocol itself. In this work, we craft a practical backdoor attack on FL systems that is proved to be simultaneously effective and stealthy on diverse use cases of FL systems and leading commercial FL platforms in the real world. Basically, we first identify a small number of redundant neurons which tend to be rarely or slightly updated in the model, and then inject backdoor into these redundant neurons instead of the whole model. In this way, our backdoor attack can achieve a high attack success rate with a minor impact on the accuracy of the original task. As countermeasures, we further consider several common technical choices including robust aggregation mechanisms, differential privacy mechanism,s and network pruning. However, none of the defenses show desirable defense capability against our backdoor attack. Our results strongly highlight the vulnerability of existing FL systems against backdoor attacks and the urgent need to develop more effective defense mechanisms.

SBPA: Sybil-Based Backdoor Poisoning Attacks for Distributed Big Data in AIoT-Based Federated Learning System

Backdoor Attacks and Defenses in Federated Learning: State-of-the-Art, Taxonomy, and Future Directions

Oblivion: Poisoning Federated Learning by Inducing Catastrophic Forgetting.

Data Poisoning Attacks Against Federated Learning Systems

BadSFL: Backdoor Attack against Scaffold Federated Learning

Shadow Backdoor Attack: Multi-intensity Backdoor Attack Against Federated Learning

How To Backdoor Federated Learning

Act in Collusion: A Persistent Distributed Multi-Target Backdoor in Federated Learning

Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey

Backdoor Federated Learning by Poisoning Backdoor-Critical Layers

Data Poisoning Attacks on Federated Machine Learning

Denial-of-Service or Fine-Grained Control: Towards Flexible Model Poisoning Attacks on Federated Learning

Towards Practical Backdoor Attacks on Federated Learning Systems

FL-PTD: A Privacy Preserving Defense Strategy Against Poisoning Attacks in Federated Learning.

Concealing Backdoor Model Updates in Federated Learning by Trigger-Optimized Data Poisoning

Protect Federated Learning Against Backdoor Attacks via Data-Free Trigger Generation

A Learning-Based Attack Framework to Break SOTA Poisoning Defenses in Federated Learning

Multitentacle Federated Learning Over Software-Defined Industrial Internet of Things Against Adaptive Poisoning Attacks

Backdoor Attacks in Peer-to-Peer Federated Learning

SPFL: A Self-purified Federated Learning Method Against Poisoning Attacks

Challenges and Countermeasures of Federated Learning Data Poisoning Attack Situation Prediction