Figure it Out: Analyzing-based Jailbreak Attack on Large Language Models

Shi Lin,Rongchang Li,Xun Wang,Changting Lin,Wenpeng Xing,Meng Han

2024-08-13

Abstract:The rapid development of Large Language Models (LLMs) has brought remarkable generative capabilities across diverse tasks. However, despite the impressive achievements, these LLMs still have numerous inherent vulnerabilities, particularly when faced with jailbreak attacks. By investigating jailbreak attacks, we can uncover hidden weaknesses in LLMs and inform the development of more robust defense mechanisms to fortify their security. In this paper, we further explore the boundary of jailbreak attacks on LLMs and propose Analyzing-based Jailbreak (ABJ). This effective jailbreak attack method takes advantage of LLMs' growing analyzing and reasoning capability and reveals their underlying vulnerabilities when facing analyzing-based tasks. We conduct a detailed evaluation of ABJ across various open-source and closed-source LLMs, which achieves 94.8% attack success rate (ASR) and 1.06 attack efficiency (AE) on GPT-4-turbo-0409, demonstrating state-of-the-art attack effectiveness and efficiency. Our research highlights the importance of prioritizing and enhancing the safety of LLMs to mitigate the risks of misuse. The code is publicly available at hhttps://github.com/theshi-1128/ABJ-Attack. Warning: This paper contains examples of LLMs that might be offensive or harmful.

Cryptography and Security,Artificial Intelligence,Computation and Language,Machine Learning

What problem does this paper attempt to address?

The paper aims to explore and address the security issues of large language models (LLMs) when facing jailbreak attacks. Specifically, the researchers propose a new jailbreak attack method—Analyzing-based Jailbreak (ABJ), which leverages the increasingly enhanced analytical and reasoning capabilities of LLMs. By designing methods that can bypass existing security mechanisms, the study reveals potential vulnerabilities in LLMs when handling analytical tasks. Experimental results show that the ABJ method can achieve an attack success rate (ASR) of up to 94.8% and an attack efficiency (AE) of 1.06 on various open-source and closed-source large-scale language models, particularly excelling on the GPT-4-turbo-0409 model. Additionally, the research evaluates the effectiveness of existing defense mechanisms against ABJ and finds that current defenses are still insufficient to fully counter this new type of attack, thereby emphasizing the importance of developing more advanced defense strategies. Overall, this work not only highlights new challenges in the security of LLMs but also provides directions for future research.