Abstract:Large Language Models (LLMs) have excelled in various tasks but are still vulnerable to jailbreaking attacks, where attackers create jailbreak prompts to mislead the model to produce harmful or offensive content. Current jailbreak methods either rely heavily on manually crafted templates, which pose challenges in scalability and adaptability, or struggle to generate semantically coherent prompts, making them easy to detect. Additionally, most existing approaches involve lengthy prompts, leading to higher query <a class="link-external link-http" href="http://costs.In" rel="external noopener nofollow">this http URL</a> this paper, to remedy these challenges, we introduce a novel jailbreaking attack framework, which is an automated, black-box jailbreaking attack framework that adapts the black-box fuzz testing approach with a series of customized designs. Instead of relying on manually crafted templates, our method starts with an empty seed pool, removing the need to search for any related jailbreaking templates. We also develop three novel question-dependent mutation strategies using an LLM helper to generate prompts that maintain semantic coherence while significantly reducing their length. Additionally, we implement a two-level judge module to accurately detect genuine successful jailbreaks. We evaluated our method on 7 representative LLMs and compared it with 5 state-of-the-art jailbreaking attack strategies. For proprietary LLM APIs, such as GPT-3.5 turbo, GPT-4, and Gemini-Pro, our method achieves attack success rates of over 90%,80% and 74%, respectively, exceeding existing baselines by more than 60%. Additionally, our method can maintain high semantic coherence while significantly reducing the length of jailbreak prompts. When targeting GPT-4, our method can achieve over 78% attack success rate even with 100 tokens. Moreover, our method demonstrates transferability and is robust to state-of-the-art defenses. We will open-source our codes upon publication.

ObscurePrompt: Jailbreaking Large Language Models via Obscure Input

"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

Jailbreaking Proprietary Large Language Models using Word Substitution Cipher

Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains

Jailbreaking Black Box Large Language Models in Twenty Queries

Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia

DROJ: A Prompt-Driven Attack against Large Language Models

On Large Language Models’ Resilience to Coercive Interrogation

Effective and Evasive Fuzz Testing-Driven Jailbreaking Attacks against LLMs

Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models

Distract Large Language Models for Automatic Jailbreak Attack

DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers

Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation

Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis

Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology

Tastle: Distract Large Language Models for Automatic Jailbreak Attack

Jailbreaking Attack against Multimodal Large Language Model

Jailbreak Attacks and Defenses Against Large Language Models: A Survey