Abstract:Although pre-trained language models (PrLMs) have achieved significant success, recent studies demonstrate that PrLMs are vulnerable to adversarial attacks. By generating adversarial examples with slight perturbations on different levels (sentence / word / character), adversarial attacks can fool PrLMs to generate incorrect predictions, which questions the robustness of PrLMs. However, we find that most existing textual adversarial examples are unnatural, which can be easily distinguished by both human and machine. Based on a general anomaly detector, we propose a novel metric (Degree of Anomaly) as a constraint to enable current adversarial attack approaches to generate more natural and imperceptible adversarial examples. Under this new constraint, the success rate of existing attacks drastically decreases, which reveals that the robustness of PrLMs is not as fragile as they claimed. In addition, we find that four types of randomization can invalidate a large portion of textual adversarial examples. Based on anomaly detector and randomization, we design a universal defense framework, which is among the first to perform textual adversarial defense without knowing the specific attack. Empirical results show that our universal defense framework achieves comparable or even higher after-attack accuracy with other specific defenses, while preserving higher original accuracy at the same time. Our work discloses the essence of textual adversarial attacks, and indicates that (1) further works of adversarial attacks should focus more on how to overcome the detection and resist the randomization, otherwise their adversarial examples would be easily detected and invalidated; and (2) compared with the unnatural and perceptible adversarial examples, it is those undetectable adversarial examples that pose real risks for PrLMs and require more attention for future robustness-enhancing strategies.

Edit-based Adversarial Text Attack

Textual adversarial attacks by exchanging text‐self words

Misleading Sentiment Analysis: Generating Adversarial Texts by the Ensemble Word Addition Algorithm

Towards Variable-Length Textual Adversarial Attacks

Textual Adversarial Attack As Combinatorial Optimization

Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations

TextCheater: A Query-Efficient Textual Adversarial Attack in the Hard-Label Setting

BESA: BERT-based Simulated Annealing for Adversarial Text Attacks.

Towards Improving Adversarial Training of NLP Models

BERT-ATTACK: Adversarial Attack Against BERT Using BERT

Semantic-Preserving Adversarial Text Attacks

Word-level Textual Adversarial Attacking as Combinatorial Optimization

Effective and Imperceptible Adversarial Textual Attack via Multi-objectivization

BFS2Adv: Black-Box Adversarial Attack Towards Hard-to-Attack Short Texts

SemAttack: Natural Textual Attacks via Different Semantic Spaces

ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model

Frauds Bargain Attack: Generating Adversarial Text Samples via Word Manipulation Process

Detecting Textual Adversarial Examples Based on Distributional Characteristics of Data Representations

Rethinking Textual Adversarial Defense for Pre-trained Language Models

Identifying Adversarial Attacks on Text Classifiers

Efficiently generating sentence-level textual adversarial examples with Seq2seq Stacked Auto-Encoder