Abstract:Backdoor attacks aim to inject backdoors to victim machine learning models during training time, such that the backdoored model maintains the prediction power of the original model towards clean inputs and misbehaves towards backdoored inputs with the trigger. The reason for backdoor attacks is that resource-limited users usually download sophisticated models from model zoos or query the models from MLaaS rather than training a model from scratch, thus a malicious third party has a chance to provide a backdoored model. In general, the more precious the model provided (i.e., models trained on rare datasets), the more popular it is with users. In this article, from a malicious model provider perspective, we propose a black-box backdoor attack, named B 3 , where neither the rare victim model (including the model architecture, parameters, and hyperparameters) nor the training data is available to the adversary. To facilitate backdoor attacks in the black-box scenario, we design a cost-effective model extraction method that leverages a carefully constructed query dataset to steal the functionality of the victim model with a limited budget. As the trigger is key to successful backdoor attacks, we develop a novel trigger generation algorithm that intensifies the bond between the trigger and the targeted misclassification label through the neuron with the highest impact on the targeted label. Extensive experiments have been conducted on various simulated deep learning models and the commercial API of Alibaba Cloud Compute Service. We demonstrate that B 3 has a high attack success rate and maintains high prediction accuracy for benign inputs. It is also shown that B 3 is robust against state-of-the-art defense strategies against backdoor attacks, such as model pruning and NC.

The triggers that open the NLP model backdoors are hidden in the adversarial samples

Training NLI Models Through Universal Adversarial Attack

B3: Backdoor Attacks Against Black-box Machine Learning Models

Hidden Backdoors in Human-Centric Language Models

BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models

Hidden Trigger Backdoor Attack on NLP Models via Linguistic Style Manipulation

Neural Network Backdoor Attacks Fully Controlled by Composite Natural Utterance Fragments.

Backdoor Attacks with Input-unique Triggers in NLP

Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor

Backdoor Pre-trained Models Can Transfer to All

Be Careful With Writing Perturbations: This Could Be a Trigger of Textual Backdoor Attacks

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

BDMMT: Backdoor Sample Detection for Language Models through Model Mutation Testing

Imperceptible Sample-Specific Backdoor to DNN with Denoising Autoencoder

Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization

Beating Backdoor Attack at Its Own Game

Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-level Backdoor Attacks

BDDR: An Effective Defense Against Textual Backdoor Attacks

Imperceptible Backdoor Attack: from Input Space to Feature Representation

Rethinking Stealthiness of Backdoor Attack Against NLP Models.

Triggerless Backdoor Attack for NLP Tasks with Clean Labels