Abstract:Multimodal contrastive learning uses various data modalities to create high-quality features, but its reliance on extensive data sources on the Internet makes it vulnerable to backdoor attacks. These attacks insert malicious behaviors during training, which are activated by specific triggers during inference, posing significant security risks. Despite existing countermeasures through fine-tuning that reduce the malicious impacts of such attacks, these defenses frequently necessitate extensive training time and degrade clean accuracy. In this study, we propose an efficient defense mechanism against backdoor threats using a concept known as machine unlearning. This entails strategically creating a small set of poisoned samples to aid the model's rapid unlearning of backdoor vulnerabilities, known as Unlearn Backdoor Threats (UBT). We specifically use overfit training to improve backdoor shortcuts and accurately detect suspicious samples in the potential poisoning data set. Then, we select fewer unlearned samples from suspicious samples for rapid forgetting in order to eliminate the backdoor effect and thus improve backdoor defense efficiency. In the backdoor unlearning process, we present a novel token-based portion unlearning training regime. This technique focuses on the model's compromised elements, dissociating backdoor correlations while maintaining the model's overall integrity. Extensive experimental results show that our method effectively defends against various backdoor attack methods in the CLIP model. Compared to SoTA backdoor defense methods, UBT achieves the lowest attack success rate while maintaining a high clean accuracy of the model (attack success rate decreases by 19% compared to SOTA, while clean accuracy increases by 2.57%).

Backdoor Contrastive Learning via Bi-level Trigger Optimization

KerbNet: A QoE-aware Kernel-Based Backdoor Attack Framework

BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning

On the Difficulty of Defending Contrastive Learning against Backdoor Attacks

Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats

BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP

Model-Contrastive Learning for Backdoor Elimination

Adversarial Backdoor Defense in CLIP

Shortcuts Arising from Contrast: Effective and Covert Clean-Label Attacks in Prompt-Based Learning

BDetCLIP: Multimodal Prompting Contrastive Test-Time Backdoor Detection

NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learning

Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning

Shortcuts Arising from Contrast: Towards Effective and Lightweight Clean-Label Attacks in Prompt-Based Learning

CleanerCLIP: Fine-grained Counterfactual Semantic Augmentation for Backdoor Defense in Contrastive Learning

LSP Framework: A Compensatory Model for Defeating Trigger Reverse Engineering via Label Smoothing Poisoning

Poisoning and Backdooring Contrastive Learning

CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning

Stand-in Backdoor: A Stealthy and Powerful Backdoor Attack

Manipulating Pre-Trained Encoder for Targeted Poisoning Attacks in Contrastive Learning

An Effective and Resilient Backdoor Attack Framework against Deep Neural Networks and Vision Transformers

UOR: Universal Backdoor Attacks on Pre-trained Language Models