ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs

Fengqing Jiang,Zhangchen Xu,Luyao Niu,Zhen Xiang,Bhaskar Ramasubramanian,Bo Li,Radha Poovendran

2024-06-08

Abstract:Safety is critical to the usage of large language models (LLMs). Multiple techniques such as data filtering and supervised fine-tuning have been developed to strengthen LLM safety. However, currently known techniques presume that corpora used for safety alignment of LLMs are solely interpreted by semantics. This assumption, however, does not hold in real-world applications, which leads to severe vulnerabilities in LLMs. For example, users of forums often use ASCII art, a form of text-based art, to convey image information. In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics. We show that five SOTA LLMs (GPT-3.5, GPT-4, Gemini, Claude, and Llama2) struggle to recognize prompts provided in the form of ASCII art. Based on this observation, we develop the jailbreak attack ArtPrompt, which leverages the poor performance of LLMs in recognizing ASCII art to bypass safety measures and elicit undesired behaviors from LLMs. ArtPrompt only requires black-box access to the victim LLMs, making it a practical attack. We evaluate ArtPrompt on five SOTA LLMs, and show that ArtPrompt can effectively and efficiently induce undesired behaviors from all five LLMs. Our code is available at <a class="link-external link-https" href="https://github.com/uw-nsl/ArtPrompt" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the security vulnerabilities present in large language models (LLMs). Specifically, existing techniques (such as data filtering and supervised fine-tuning) assume that the corpus used to align LLM security is interpreted solely through semantics. However, in practical applications, this assumption does not always hold, leading to security vulnerabilities in LLMs. For example, forum users often use ASCII art to convey image information, and existing LLMs often fail to correctly recognize these forms of information. Therefore, the paper proposes an ASCII art-based jailbreak attack method—ArtPrompt, and introduces a comprehensive benchmark test, the Vision-in-Text Challenge (VITC), to evaluate the ability of current state-of-the-art LLMs in handling non-semantic prompts. The research shows that five state-of-the-art LLMs, including GPT-3.5, GPT-4, Gemini, Claude, and Llama2, have difficulty recognizing prompts in the form of ASCII art. ArtPrompt exploits this weakness to bypass security measures and induce LLMs to produce unsafe behavior. Additionally, ArtPrompt only requires black-box access to the target LLM, making it a practical and efficient attack method.

ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs

Tastle: Distract Large Language Models for Automatic Jailbreak Attack

Effective and Evasive Fuzz Testing-Driven Jailbreaking Attacks against LLMs

Feint and Attack: Attention-Based Strategies for Jailbreaking and Protecting LLMs

IDEATOR: Jailbreaking VLMs Using VLMs

Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation

FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts

"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts

Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt

AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs

Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis

An LLM can Fool Itself: A Prompt-Based Adversarial Attack

IDEATOR: Jailbreaking Large Vision-Language Models Using Themselves

h4rm3l: A Dynamic Benchmark of Composable Jailbreak Attacks for LLM Safety Assessment

Jailbreaking Text-to-Image Models with LLM-Based Agents

Jailbreaking Proprietary Large Language Models using Word Substitution Cipher

DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers

White-box Multimodal Jailbreaks Against Large Vision-Language Models

PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach

ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings