Abstract:Large Language Models (LLMs) have excelled in various tasks but are still vulnerable to jailbreaking attacks, where attackers create jailbreak prompts to mislead the model to produce harmful or offensive content. Current jailbreak methods either rely heavily on manually crafted templates, which pose challenges in scalability and adaptability, or struggle to generate semantically coherent prompts, making them easy to detect. Additionally, most existing approaches involve lengthy prompts, leading to higher query <a class="link-external link-http" href="http://costs.In" rel="external noopener nofollow">this http URL</a> this paper, to remedy these challenges, we introduce a novel jailbreaking attack framework, which is an automated, black-box jailbreaking attack framework that adapts the black-box fuzz testing approach with a series of customized designs. Instead of relying on manually crafted templates, our method starts with an empty seed pool, removing the need to search for any related jailbreaking templates. We also develop three novel question-dependent mutation strategies using an LLM helper to generate prompts that maintain semantic coherence while significantly reducing their length. Additionally, we implement a two-level judge module to accurately detect genuine successful jailbreaks. We evaluated our method on 7 representative LLMs and compared it with 5 state-of-the-art jailbreaking attack strategies. For proprietary LLM APIs, such as GPT-3.5 turbo, GPT-4, and Gemini-Pro, our method achieves attack success rates of over 90%,80% and 74%, respectively, exceeding existing baselines by more than 60%. Additionally, our method can maintain high semantic coherence while significantly reducing the length of jailbreak prompts. When targeting GPT-4, our method can achieve over 78% attack success rate even with 100 tokens. Moreover, our method demonstrates transferability and is robust to state-of-the-art defenses. We will open-source our codes upon publication.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper primarily focuses on the security issues of large language models (LLMs) when faced with jailbreak attacks. Specifically, the researchers propose a new automated, black-box jailbreak attack framework to overcome the limitations of existing methods and improve the success rate and efficiency of attacks. #### Main Issues 1. **Dependence on Manual Templates**: Existing jailbreak methods either rely on manually designed templates, which pose challenges in scalability and adaptability, or struggle to generate semantically coherent prompts, making them easily detectable. 2. **Excessive Prompt Length**: Most existing methods generate lengthy prompts, leading to high query costs and potential alarm triggers. 3. **Semantic Coherence and Transferability**: Maintaining the semantic coherence of prompts and their effectiveness across different models is crucial for evading defenses, but existing methods perform poorly in this regard. 4. **Evasion of Existing Defenses**: As defenses against jailbreak attacks continue to increase, attackers need to find ways to circumvent these measures to maintain the effectiveness of their attacks. #### Solutions 1. **No Dependence on Existing Templates**: This method starts from an empty seed pool and does not rely on any existing jailbreak templates, thereby enhancing the practicality and efficiency of the attack. 2. **Generation of Short and Meaningful Prompts**: By using three LLM-based mutation strategies, the method generates semantically coherent and concise prompts, significantly reducing attack costs. 3. **Accurate Identification of Successful Jailbreak Prompts**: A two-layer judgment module is employed to accurately identify truly successful jailbreaks, including illegal content detection and response matching verification. 4. **Experimental Validation**: Extensive experiments were conducted on multiple representative LLM models, and compared to five state-of-the-art jailbreak attack strategies, this method showed significant improvements in both success rate and efficiency. In summary, the paper aims to develop a novel jailbreak attack framework that can automatically and efficiently generate short and semantically coherent jailbreak prompts while effectively evading existing defenses.

Effective and Evasive Fuzz Testing-Driven Jailbreaking Attacks against LLMs

Tastle: Distract Large Language Models for Automatic Jailbreak Attack

Comprehensive Assessment of Jailbreak Attacks Against LLMs

Distract Large Language Models for Automatic Jailbreak Attack

GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts

Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs

JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs

EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models

Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach

Jailbreak Attacks and Defenses Against Large Language Models: A Survey

Jailbreaking Black Box Large Language Models in Twenty Queries

Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks

AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

Lockpicking LLMs: A Logit-Based Jailbreak Using Token-level Manipulation

Figure it Out: Analyzing-based Jailbreak Attack on Large Language Models

Tree of Attacks: Jailbreaking Black-Box LLMs Automatically

Model-Editing-Based Jailbreak against Safety-aligned Large Language Models

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

h4rm3l: A Dynamic Benchmark of Composable Jailbreak Attacks for LLM Safety Assessment