Abstract:Large Language Models (LLMs) are increasingly being developed and applied, but their widespread use faces challenges. These include aligning LLMs' responses with human values to prevent harmful outputs, which is addressed through safety training methods. Even so, bad actors and malicious users have succeeded in attempts to manipulate the LLMs to generate misaligned responses for harmful questions such as methods to create a bomb in school labs, recipes for harmful drugs, and ways to evade privacy rights. Another challenge is the multilingual capabilities of LLMs, which enable the model to understand and respond in multiple languages. Consequently, attackers exploit the unbalanced pre-training datasets of LLMs in different languages and the comparatively lower model performance in low-resource languages than high-resource ones. As a result, attackers use a low-resource languages to intentionally manipulate the model to create harmful responses. Many of the similar attack vectors have been patched by model providers, making the LLMs more robust against language-based manipulation. In this paper, we introduce a new black-box attack vector called the \emph{Sandwich attack}: a multi-language mixture attack, which manipulates state-of-the-art LLMs into generating harmful and misaligned responses. Our experiments with five different models, namely Google's Bard, Gemini Pro, LLaMA-2-70-B-Chat, GPT-3.5-Turbo, GPT-4, and Claude-3-OPUS, show that this attack vector can be used by adversaries to generate harmful responses and elicit misaligned responses from these models. By detailing both the mechanism and impact of the Sandwich attack, this paper aims to guide future research and development towards more secure and resilient LLMs, ensuring they serve the public good while minimizing potential for misuse.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the security issue of large - language models (LLMs) in a multilingual environment. Specifically, although existing LLMs have been trained for security to prevent the generation of harmful outputs, malicious users can still manipulate these models to generate harmful responses that are inconsistent with human values through specific attack methods (such as jailbreak attacks, multilingual injection, etc.). In addition, due to the imbalance of pre - training data among different languages and the poor performance of low - resource languages, attackers can take advantage of this to manipulate models to generate harmful content. To solve these problems, the author introduced a new black - box attack method - "Sandwich attack". This attack method induces LLMs to generate harmful and inconsistent responses by writing a series of questions in different low - resource languages and hiding malicious questions in the middle position. The author demonstrated the effectiveness of this attack through experiments on five different advanced LLMs and revealed the vulnerabilities of current LLMs in multilingual adaptation and security mechanisms. The following is a summary of the key points of this paper: 1. **Research background**: - The security of LLMs is facing challenges, especially in a multilingual environment. - Malicious users can manipulate LLMs to generate harmful content through various means (such as jailbreak attacks). - The imbalance of pre - training data in different languages makes low - resource languages more vulnerable to attacks. 2. **Proposed questions**: - How to prevent malicious users from manipulating LLMs to generate harmful content through a multilingual environment? - Is the existing security training sufficient to deal with complex multilingual attacks? 3. **Solutions**: - Introduce the "Sandwich attack" method to hide malicious requests through multilingual mixed questions. - Evaluate the effectiveness of this attack method through experiments on five advanced LLMs. 4. **Contributions**: - Discover a new general - purpose black - box attack method - "Sandwich attack". - Empirically demonstrate the self - evaluation failure of existing LLMs in a multilingual mixed setting. - Observe and record the behavior patterns of LLMs under the "Sandwich attack". - Provide an empirical analysis that the security mechanisms of LLMs rely on English texts rather than other non - English texts. Through these studies, the author aims to guide future research and development to improve the security and robustness of LLMs, ensuring that they can better serve the public interest while reducing the potential risk of abuse.

Sandwich attack: Multi-language Mixture Adaptive Attack on LLMs

Universal and Transferable Adversarial Attacks on Aligned Language Models

Open Sesame! Universal Black Box Jailbreaking of Large Language Models

Multilingual Jailbreak Challenges in Large Language Models

Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing

LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked

PAL: Proxy-Guided Black-Box Attack on Large Language Models

Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes

Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models

Can LLMs be Fooled? Investigating Vulnerabilities in LLMs

Imposter.AI: Adversarial Attacks with Hidden Intentions towards Aligned Large Language Models

Misusing Tools in Large Language Models With Visual Adversarial Examples

Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content

Adversarial Attacks and Defenses in Large Language Models: Old and New Threats

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

Vocabulary Attack to Hijack Large Language Model Applications

Jailbreaking and Mitigation of Vulnerabilities in Large Language Models

Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation

Exploring the Adversarial Capabilities of Large Language Models

Baseline Defenses for Adversarial Attacks Against Aligned Language Models