Abstract:Although pre-trained language models (PrLMs) have achieved significant success, recent studies demonstrate that PrLMs are vulnerable to adversarial attacks. By generating adversarial examples with slight perturbations on different levels (sentence / word / character), adversarial attacks can fool PrLMs to generate incorrect predictions, which questions the robustness of PrLMs. However, we find that most existing textual adversarial examples are unnatural, which can be easily distinguished by both human and machine. Based on a general anomaly detector, we propose a novel metric (Degree of Anomaly) as a constraint to enable current adversarial attack approaches to generate more natural and imperceptible adversarial examples. Under this new constraint, the success rate of existing attacks drastically decreases, which reveals that the robustness of PrLMs is not as fragile as they claimed. In addition, we find that four types of randomization can invalidate a large portion of textual adversarial examples. Based on anomaly detector and randomization, we design a universal defense framework, which is among the first to perform textual adversarial defense without knowing the specific attack. Empirical results show that our universal defense framework achieves comparable or even higher after-attack accuracy with other specific defenses, while preserving higher original accuracy at the same time. Our work discloses the essence of textual adversarial attacks, and indicates that (1) further works of adversarial attacks should focus more on how to overcome the detection and resist the randomization, otherwise their adversarial examples would be easily detected and invalidated; and (2) compared with the unnatural and perceptible adversarial examples, it is those undetectable adversarial examples that pose real risks for PrLMs and require more attention for future robustness-enhancing strategies.

Text Laundering: Mitigating Malicious Features Through Knowledge Distillation of Large Foundation Models.

Kallima: A Clean-Label Framework for Textual Backdoor Attacks

BDDR: An Effective Defense Against Textual Backdoor Attacks

Neutralizing Backdoors through Information Conflicts for Large Language Models

Defensive Dual Masking for Robust Adversarial Defense

Unlearning Backdoor Attacks for LLMs with Weak-to-Strong Knowledge Distillation

Adversarial Text Purification: A Large Language Model Approach for Defense

Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks

Expose Backdoors on the Way: A Feature-Based Efficient Defense Against Textual Backdoor Attacks

Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors

Towards a Robust Deep Neural Network Against Adversarial Texts: A Survey.

Defense Against Syntactic Textual Backdoor Attacks with Token Substitution

Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey

Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack

Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks

Enhanced Attacks on Defensively Distilled Deep Neural Networks.

Towards a Robust Deep Neural Network in Texts: A Survey

Rethinking Textual Adversarial Defense for Pre-trained Language Models

Red Teaming Language Model Detectors with Language Models

Searching for an Effective Defender: Benchmarking Defense Against Adversarial Word Substitution

BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models