Abstract:In this paper, we present a simple yet surprisingly effective technique to induce "selective amnesia" on a backdoored model. Our approach, called SEAM, has been inspired by the problem of catastrophic forgetting (CF), a long standing issue in continual learning. Our idea is to retrain a given DNN model on randomly labeled clean data, to induce a CF on the model, leading to a sudden forget on both primary and backdoor tasks; then we recover the primary task by retraining the randomized model on correctly labeled clean data. We analyzed SEAM by modeling the unlearning process as continual learning and further approximating a DNN using Neural Tangent Kernel for measuring CF. Our analysis shows that our random-labeling approach actually maximizes the CF on an unknown backdoor in the absence of triggered inputs, and also preserves some feature extraction in the network to enable a fast revival of the primary task. We further evaluated SEAM on both image processing and Natural Language Processing tasks, under both data contamination and training manipulation attacks, over thousands of models either trained on popular image datasets or provided by the TrojAI competition. Our experiments show that SEAM vastly outperforms the state-of-the-art unlearning techniques, achieving a high Fidelity (measuring the gap between the accuracy of the primary task and that of the backdoor) within a few minutes (about 30 times faster than training a model from scratch using the MNIST dataset), with only a small amount of clean data (0.1% of training data for TrojAI models).

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to remove the backdoor effect in deep neural networks (DNN) efficiently and with high - fidelity. Specifically, the paper targets the hidden backdoor tasks injected through training data contamination or training process manipulation in DNN models. These backdoor tasks will cause the model to strategically misclassify the input when a specific trigger is detected. Traditional defense methods either need to recover the trigger, which is very difficult in practice, or try to fine - tune the model with a small amount of clean data, but this method has poor performance when clean data is scarce. Therefore, the paper proposes a new technique - Selective Erasure of Adversarial Manipulation (SEAM), aiming to effectively remove the backdoor effect from the infected model without trigger information while maintaining the main task performance of the model. ### Key problems solved by the paper: 1. **Efficient backdoor removal**: Existing methods usually require a large amount of clean data or specific trigger information, while SEAM can quickly and efficiently remove the backdoor effect with only a small amount of clean data. 2. **High - fidelity**: SEAM can not only effectively remove the backdoor but also maintain the high performance of the model on the main task, that is, high - fidelity (Fidelity). 3. **Blind removal**: SEAM is a blind removal technique. It does not need to know the specific trigger information and is suitable for the case of unknown backdoors. ### Main contributions: - **Novel backdoor defense technique**: Use Catastrophic Forgetting (CF) to achieve efficient backdoor removal. - **Theoretical analysis**: Explain the effectiveness of SEAM through multi - task learning and Neural Tangent Kernel (NTK) theory. - **Extensive experimental verification**: Conduct a large number of experiments on DNN models of multiple architectures to verify the effectiveness of SEAM on different tasks and datasets, especially in image processing and natural language processing tasks. ### Technical details: - **Selective Erasure of Adversarial Manipulation (SEAM)**: - **Forgetting phase**: Retrain the model on a dataset with random labels to induce catastrophic forgetting, making the model forget the main task and the backdoor task. - **Recovery phase**: Retrain the model with a small amount of clean data to recover the performance of the main task without awakening the backdoor task. - **Theoretical analysis**: - **Multi - task learning modeling**: Model the backdoor attack as a multi - task learning problem and analyze the relationship between the main task and the backdoor task. - **Neural Tangent Kernel (NTK)**: Use NTK theory to approximate DNN models, measure the impact of catastrophic forgetting, and explain why the random - label task can maximize the effect of forgetting the backdoor task. ### Experimental results: - **Efficiency**: On the MNIST dataset, SEAM only needs about 30 times the time required for training from scratch to achieve high - fidelity. - **Small data requirement**: Only 0.1% of the training data is required to completely suppress the backdoor effect. - **Outperforming existing methods**: On multiple datasets and tasks, the performance of SEAM far exceeds that of the existing state - of - the - art backdoor removal techniques, such as Neural Cleanse, Fine - Pruning, and Neural Attention Distillation. In conclusion, this paper proposes an innovative backdoor defense technique SEAM, which realizes efficient and high - fidelity backdoor removal by using catastrophic forgetting and has important theoretical and practical significance.

Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models

ATTEQ-NN: Attention-based QoE-aware Evasive Backdoor Attacks.

Redeem Myself: Purifying Backdoors in Deep Learning Models Using Self Attention Distillation.

Overcoming Long-Term Catastrophic Forgetting Through Adversarial Neural Pruning and Synaptic Consolidation

Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats

Backdoor Defense with Machine Unlearning

Need for Speed: Taming Backdoor Attacks with Speed and Precision

Persistent Backdoor Attacks in Continual Learning

Mitigating Backdoor Attacks using Activation-Guided Model Editing

Memory Backdoor Attacks on Neural Networks

Reverse Engineering Imperceptible Backdoor Attacks on Deep Neural Networks for Detection and Training Set Cleansing

Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness

Flatness-aware Sequential Learning Generates Resilient Backdoors

Unified Neural Backdoor Removal with Only Few Clean Samples through Unlearning and Relearning

Unlearn to Relearn Backdoors: Deferred Backdoor Functionality Attacks on Deep Learning Models

MEDIC: Remove Model Backdoors via Importance Driven Cloning

Anti-Backdoor Learning: Training Clean Models on Poisoned Data

Backdoor Cleansing with Unlabeled Data

Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning

Design of intentional backdoors in sequential models

Progressive Backdoor Erasing Via Connecting Backdoor and Adversarial Attacks