Abstract:Despite the general capabilities of Large Language Models (LLM), these models still request fine-tuning or adaptation with customized data when meeting specific business demands. However, this process inevitably introduces new threats, particularly against the Fine-tuning based Jailbreak Attack (FJAttack) under the setting of Language-Model-as-a-Service (LMaaS), where the model's safety has been significantly compromised by fine-tuning users' uploaded examples contain just a few harmful examples. Though potential defenses have been proposed that the service providers can integrate safety examples into the fine-tuning dataset to reduce safety issues, such approaches require incorporating a substantial amount of data, making it inefficient. To effectively defend against the FJAttack with limited safety examples under LMaaS, we propose the Backdoor Enhanced Safety Alignment method inspired by an analogy with the concept of backdoor attacks. In particular, service providers will construct prefixed safety examples with a secret prompt, acting as a "backdoor trigger". By integrating prefixed safety examples into the fine-tuning dataset, the subsequent fine-tuning process effectively acts as the "backdoor attack", establishing a strong correlation between the secret prompt and safety generations. Consequently, safe responses are ensured once service providers prepend this secret prompt ahead of any user input during inference. Our comprehensive experiments demonstrate that through the Backdoor Enhanced Safety Alignment with adding as few as 11 prefixed safety examples, the maliciously fine-tuned LLMs will achieve similar safety performance as the original aligned models without harming the benign performance. Furthermore, we also present the effectiveness of our method in a more practical setting where the fine-tuning data consists of both FJAttack examples and the fine-tuning task data.

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates

Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B

Locking Down the Finetuned LLMs Safety

Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning

Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes

What Makes and Breaks Safety Fine-tuning? A Mechanistic Study

Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment

NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection

Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning

She had Cobalt Blue Eyes: Prompt Testing to Create Aligned and Sustainable Language Models

Safety Alignment Should Be Made More Than Just a Few Tokens Deep

Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models

Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses

ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging

Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases

Mimicking User Data: On Mitigating Fine-Tuning Risks in Closed Large Language Models

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation

Fight Back Against Jailbreaking via Prompt Adversarial Tuning