Abstract:The swift advancement of large language models (LLMs) has profoundly shaped the landscape of artificial intelligence; however, their deployment in sensitive domains raises grave concerns, particularly due to their susceptibility to malicious exploitation. This situation underscores the insufficiencies in pre-deployment testing, highlighting the urgent need for more rigorous and comprehensive evaluation methods. This study presents a comprehensive empirical analysis assessing the efficacy of conventional coverage criteria in identifying these vulnerabilities, with a particular emphasis on the pressing issue of jailbreak attacks. Our investigation begins with a clustering analysis of the hidden states in LLMs, demonstrating that intrinsic characteristics of these states can distinctly differentiate between various types of queries. Subsequently, we assess the performance of these criteria across three critical dimensions: criterion level, layer level, and token level. Our findings uncover significant disparities in neuron activation patterns between the processing of normal and jailbreak queries, thereby corroborating the clustering results. Leveraging these findings, we propose an innovative approach for the real-time detection of jailbreak attacks by utilizing neural activation features. Our classifier demonstrates remarkable accuracy, averaging 96.33% in identifying jailbreak queries, including those that could lead to adversarial attacks. The importance of our research lies in its comprehensive approach to addressing the intricate challenges of LLM security. By enabling instantaneous detection from the model's first token output, our method holds promise for future systems integrating LLMs, offering robust real-time detection capabilities. This study advances our understanding of LLM security testing, and lays a critical foundation for the development of more resilient AI systems.

A Cross-Language Investigation into Jailbreak Attacks in Large Language Models

Jailbreak Attacks and Defenses Against Large Language Models: A Survey

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective

JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks

Jailbreaking and Mitigation of Vulnerabilities in Large Language Models

Figure it Out: Analyzing-based Jailbreak Attack on Large Language Models

Distract Large Language Models for Automatic Jailbreak Attack

Comprehensive Assessment of Jailbreak Attacks Against LLMs

Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction

Multilingual Jailbreak Challenges in Large Language Models

Investigating Coverage Criteria in Large Language Models: An In-Depth Study Through Jailbreak Attacks

Improved Large Language Model Jailbreak Detection via Pretrained Embeddings

PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach

Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Models

Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models

Playing Language Game with LLMs Leads to Jailbreaking

Jailbreaking LLMs with Arabic Transliteration and Arabizi

Jailbreaking Attack against Multimodal Large Language Model

Hide Your Malicious Goal Into Benign Narratives: Jailbreak Large Language Models through Neural Carrier Articles