Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring

Honglin Mu,Han He,Yuxin Zhou,Yunlong Feng,Yang Xu,Libo Qin,Xiaoming Shi,Zeming Liu,Xudong Han,Qi Shi,Qingfu Zhu,Wanxiang Che

2024-10-28

Abstract:Large language model (LLM) safety is a critical issue, with numerous studies employing red team testing to enhance model security. Among these, jailbreak methods explore potential vulnerabilities by crafting malicious prompts that induce model outputs contrary to safety alignments. Existing black-box jailbreak methods often rely on model feedback, repeatedly submitting queries with detectable malicious instructions during the attack search process. Although these approaches are effective, the attacks may be intercepted by content moderators during the search process. We propose an improved transfer attack method that guides malicious prompt construction by locally training a mirror model of the target black-box model through benign data distillation. This method offers enhanced stealth, as it does not involve submitting identifiable malicious instructions to the target model during the search phase. Our approach achieved a maximum attack success rate of 92%, or a balanced value of 80% with an average of 1.5 detectable jailbreak queries per sample against GPT-3.5 Turbo on a subset of AdvBench. These results underscore the need for more robust defense mechanisms.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the security issues of large language models (LLMs), particularly focusing on "jailbreak attacks." These attacks involve crafting malicious prompts to induce the model to generate outputs that violate safety regulations. Existing black-box jailbreak methods, although effective, require repeatedly submitting identifiable malicious commands during the attack process, which may be intercepted by content review mechanisms. Therefore, the paper proposes an improved transfer attack method called ShadowBreak, which involves locally training a mirror model of the target black-box model to guide the construction of malicious prompts. This method enhances the stealthiness of the attack by not directly submitting identifiable malicious commands to the target model during the attack process.

Specifically, the main contributions of the paper include:
1. **Defining metrics for evaluating the stealthiness of jailbreak attacks**: In addition to the traditional attack success rate (ASR), the number of queries that could be detected during the attack process is also considered.
2. **Proposing the ShadowBreak method**: By using benign data mirroring techniques, it achieves a high success rate while minimizing the number of detectable queries, thereby enhancing the stealthiness of the attack.
3. **Exposing potential vulnerabilities in current security mechanisms**: Particularly in the context of aligned transfer attacks, it emphasizes the need to develop more robust and adaptive defense strategies.

Through these contributions, the paper aims to improve the understanding of the security of large language models and promote the development of more effective defense mechanisms.