Abstract:Because "out-of-the-box" large language models are capable of generating a great deal of objectionable content, recent work has focused on aligning these models in an attempt to prevent undesirable generation. While there has been some success at circumventing these measures -- so-called "jailbreaks" against LLMs -- these attacks have required significant human ingenuity and are brittle in practice. In this paper, we propose a simple and effective attack method that causes aligned language models to generate objectionable behaviors. Specifically, our approach finds a suffix that, when attached to a wide range of queries for an LLM to produce objectionable content, aims to maximize the probability that the model produces an affirmative response (rather than refusing to answer). However, instead of relying on manual engineering, our approach automatically produces these adversarial suffixes by a combination of greedy and gradient-based search techniques, and also improves over past automatic prompt generation methods. Surprisingly, we find that the adversarial prompts generated by our approach are quite transferable, including to black-box, publicly released LLMs. Specifically, we train an adversarial attack suffix on multiple prompts (i.e., queries asking for many different types of objectionable content), as well as multiple models (in our case, Vicuna-7B and 13B). When doing so, the resulting attack suffix is able to induce objectionable content in the public interfaces to ChatGPT, Bard, and Claude, as well as open source LLMs such as LLaMA-2-Chat, Pythia, Falcon, and others. In total, this work significantly advances the state-of-the-art in adversarial attacks against aligned language models, raising important questions about how such systems can be prevented from producing objectionable information. Code is available at <a class="link-external link-http" href="http://github.com/llm-attacks/llm-attacks" rel="external noopener nofollow">this http URL</a>.

Self-Evaluation as a Defense Against Adversarial Attacks on LLMs

LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked

Large Language Model Sentinel: Advancing Adversarial Robustness by LLM Agent

Large Language Model Sentinel: LLM Agent for Adversarial Purification

An LLM can Fool Itself: A Prompt-Based Adversarial Attack

LLM Evaluators Recognize and Favor Their Own Generations

SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts

Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility

Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences

Baseline Defenses for Adversarial Attacks Against Aligned Language Models

Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment

A LLM Assisted Exploitation of AI-Guardian

Self-Guard: Empower the LLM to Safeguard Itself

Universal and Transferable Adversarial Attacks on Aligned Language Models

CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities

Defending Large Language Models Against Attacks With Residual Stream Activation Analysis

Can LLMs Patch Security Issues?

The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness

Can LLMs be Fooled? Investigating Vulnerabilities in LLMs