Abstract:Deep neural networks (DNNs) are known to be inherently vulnerable to malicious attacks such as the adversarial attack and the backdoor attack. The former is crafted by adding small perturbations to benign inputs so as to fool a DNN. The latter generally embeds a hidden pattern in a DNN by poisoning the dataset during the training process, which causes the infected model to misbehave on predefined inputs with a specific trigger and normally perform for others. Much work has been conducted on defending against the adversarial samples, while the backdoor attack received much less attention, especially in recurrent neural networks (RNNs), which play an important role in the text processing field. Two main limitations make it hard to directly apply existing image backdoor detection approaches to RNN-based text classification systems. First, a layer in an RNN does not preserve the same feature latent space function for different inputs, making it impossible to map the inserted specific pattern with the neural activations. Second, the text data is inherently discrete, making it hard to optimize the text like image pixels. In this work, we propose a novel backdoor detection approach named InterRNN for RNN-based text classification systems from the interpretation perspective. Specifically, we first propose a novel RNN interpretation technique by constructing a nondeterministic finite automaton (NFA) based abstract model, which can effectively reduce the analysis complexity of an RNN while preserving its original logic rules. Then, based on the abstract model, we can obtain interpretation results that explain the fundamental reason behind the decision for each input. We then detect trigger words by leveraging the differences between the behaviors in the backdoor sentences and those in the normal sentences. The extensive experiment results on four benchmark datasets demonstrate that our approach can generate better interpretation results compared to state-of-the-art approaches and effectively detect backdoors in RNNs.

ONION: A Simple and Effective Defense Against Textual Backdoor Attacks

BDDR: An Effective Defense Against Textual Backdoor Attacks

Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks

Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger

Expose Backdoors on the Way: A Feature-Based Efficient Defense Against Textual Backdoor Attacks

NLPSweep: A comprehensive defense scheme for mitigating NLP backdoor attacks

Towards a Robust Deep Neural Network in Texts: A Survey

Attack and Defense of Deep Learning Models in the Field of Web Attack Detection

Towards a Robust Deep Neural Network Against Adversarial Texts: A Survey.

Text Backdoor Detection Using an Interpretable RNN Abstract Model.

Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution

Textual Adversarial Attack As Combinatorial Optimization

Text Laundering: Mitigating Malicious Features Through Knowledge Distillation of Large Foundation Models.

Rethinking Stealthiness of Backdoor Attack Against NLP Models.

Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks

Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor

Beating Backdoor Attack at Its Own Game

A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks

Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey

NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learning

A backdoor attack against LSTM-based text classification systems