Abstract:Federated self-supervised learning (FSSL) has recently emerged as a promising paradigm that enables the exploitation of clients' vast amounts of unlabeled data while preserving data privacy. While FSSL offers advantages, its susceptibility to backdoor attacks, a concern identified in traditional federated supervised learning (FSL), has not been investigated. To fill the research gap, we undertake a comprehensive investigation into a backdoor attack paradigm, where unscrupulous clients conspire to manipulate the global model, revealing the vulnerability of FSSL to such attacks. In FSL, backdoor attacks typically build a direct association between the backdoor trigger and the target label. In contrast, in FSSL, backdoor attacks aim to alter the global model's representation for images containing the attacker's specified trigger pattern in favor of the attacker's intended target class, which is less straightforward. In this sense, we demonstrate that existing defenses are insufficient to mitigate the investigated backdoor attacks in FSSL, thus finding an effective defense mechanism is urgent. To tackle this issue, we dive into the fundamental mechanism of backdoor attacks on FSSL, proposing the Embedding Inspector (EmInspector) that detects malicious clients by inspecting the embedding space of local models. In particular, EmInspector assesses the similarity of embeddings from different local models using a small set of inspection images (e.g., ten images of CIFAR100) without specific requirements on sample distribution or labels. We discover that embeddings from backdoored models tend to cluster together in the embedding space for a given inspection image. Evaluation results show that EmInspector can effectively mitigate backdoor attacks on FSSL across various adversary settings. Our code is avaliable at
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the backdoor attack problem in Federated Self - Supervised Learning (FSSL). Specifically, the author points out that while FSSL utilizes a large amount of unlabeled data from clients, it also faces the threat of backdoor attacks from malicious clients. Compared with traditional Federated Supervised Learning (FSL), backdoor attacks in FSSL are more concealed and difficult to detect, because the attacker's goal is to manipulate the model representation in the high - dimensional embedding space rather than directly associate the trigger and the target label.
### Problem Background
1. **Advantages and Challenges of FSSL**:
- FSSL can use a large amount of unlabeled data from clients for model training while protecting data privacy.
- However, FSSL also inherits the vulnerability of traditional FSL to backdoor attacks, and research in this area is still insufficient.
2. **Characteristics of Backdoor Attacks**:
- In FSL, backdoor attacks mislead the global model by uploading data - label pairs with triggers.
- In FSSL, the goal of backdoor attacks is to make the model generate embedding vectors biased towards the attacker - specified target category when processing inputs with specific triggers, thus affecting the prediction results of downstream classifiers.
3. **Deficiencies of Existing Defense Mechanisms**:
- Existing defense mechanisms are mainly based on statistical features or feedback mechanisms, and these methods are not effective in FSSL, because the backdoor model can optimize the loss function to make its output very similar to that of a normal model, thus evading detection.
### Main Contributions of the Paper
1. **Revealing the Backdoor Attack Risk in FSSL**:
- The author comprehensively explores backdoor attacks in FSSL for the first time and demonstrates the effectiveness and concealment of such attacks.
2. **Proposing the EmInspector Defense Mechanism**:
- EmInspector identifies malicious clients by examining abnormal embeddings in the embedding space. It utilizes a key observation: for a given image, the embeddings generated by the backdoor - attacked local model tend to cluster together and maintain a certain similarity with the embeddings generated by the benign model.
- Experiments show that EmInspector can accurately identify malicious clients with only a small number of test images, without special requirements for sample distribution or labels.
3. **Extensive Experimental Verification**:
- The author conducts extensive experiments under various settings (such as different datasets, training data distributions, adversarial settings, network architectures, etc.), proving the effectiveness and robustness of EmInspector.
### Conclusion
This paper aims to fill the research gap in backdoor attacks in FSSL and proposes a novel and effective defense mechanism, EmInspector, to deal with this security threat. By in - depth analysis of the backdoor attack mechanism and designing a targeted detection method, EmInspector provides a reliable defense means for FSSL systems.