PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach

Zhihao Lin,Wei Ma,Mingyi Zhou,Yanjie Zhao,Haoyu Wang,Yang Liu,Jun Wang,Li Li
2024-10-03
Abstract:In recent years, Large Language Models (LLMs) have gained widespread use, raising concerns about their security. Traditional jailbreak attacks, which often rely on the model internal information or have limitations when exploring the unsafe behavior of the victim model, limiting their reducing their general applicability. In this paper, we introduce PathSeeker, a novel black-box jailbreak method, which is inspired by the game of rats escaping a maze. We think that each LLM has its unique "security maze", and attackers attempt to find the exit learning from the received feedback and their accumulated experience to compromise the target LLM's security defences. Our approach leverages multi-agent reinforcement learning, where smaller models collaborate to guide the main LLM in performing mutation operations to achieve the attack objectives. By progressively modifying inputs based on the model's feedback, our system induces richer, harmful responses. During our manual attempts to perform jailbreak attacks, we found that the vocabulary of the response of the target model gradually became richer and eventually produced harmful responses. Based on the observation, we also introduce a reward mechanism that exploits the expansion of vocabulary richness in LLM responses to weaken security constraints. Our method outperforms five state-of-the-art attack techniques when tested across 13 commercial and open-source LLMs, achieving high attack success rates, especially in strongly aligned commercial models like GPT-4o-mini, Claude-3.5, and GLM-4-air with strong safety alignment. This study aims to improve the understanding of LLM security vulnerabilities and we hope that this sturdy can contribute to the development of more robust defenses.
Cryptography and Security,Artificial Intelligence
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to address the security vulnerabilities of large language models (LLMs), particularly focusing on the limitations of existing jailbreak attack methods. Specifically, traditional jailbreak attack methods often rely on internal model information or have limitations when exploring unsafe behaviors of the victim model, which reduces their generalizability. To solve these issues, the paper proposes a new black-box jailbreak method called PathSeeker. **Main Objectives:** 1. **Propose a new jailbreak method**: PathSeeker is a novel black-box jailbreak method based on multi-agent reinforcement learning, where small models collaborate to guide the main LLM to perform mutation operations to achieve the attack goal. 2. **Enhance attack effectiveness**: By gradually modifying the input and adjusting strategies based on model feedback, the system can induce more harmful responses. 3. **Improve generalizability**: Compared to existing techniques, PathSeeker demonstrates a higher attack success rate on multiple commercial and open-source LLMs, especially in commercially strong security-aligned models (such as GPT-4o-mini, Claude-3.5, and GLM-4-air). **Specific Problems:** - How to design a jailbreak attack method that does not rely on proxy models to generate harmful reference answers? - How to utilize multi-agent reinforcement learning to improve the feedback mechanism during the attack process? - How to evaluate the success rate of the attack and validate the effectiveness of the method in experiments? Through this research, the paper hopes to deepen the understanding of LLM security vulnerabilities and promote the development of more robust defense measures.