Abstract:Despite the general capabilities of Large Language Models (LLM), these models still request fine-tuning or adaptation with customized data when meeting specific business demands. However, this process inevitably introduces new threats, particularly against the Fine-tuning based Jailbreak Attack (FJAttack) under the setting of Language-Model-as-a-Service (LMaaS), where the model's safety has been significantly compromised by fine-tuning users' uploaded examples contain just a few harmful examples. Though potential defenses have been proposed that the service providers can integrate safety examples into the fine-tuning dataset to reduce safety issues, such approaches require incorporating a substantial amount of data, making it inefficient. To effectively defend against the FJAttack with limited safety examples under LMaaS, we propose the Backdoor Enhanced Safety Alignment method inspired by an analogy with the concept of backdoor attacks. In particular, service providers will construct prefixed safety examples with a secret prompt, acting as a "backdoor trigger". By integrating prefixed safety examples into the fine-tuning dataset, the subsequent fine-tuning process effectively acts as the "backdoor attack", establishing a strong correlation between the secret prompt and safety generations. Consequently, safe responses are ensured once service providers prepend this secret prompt ahead of any user input during inference. Our comprehensive experiments demonstrate that through the Backdoor Enhanced Safety Alignment with adding as few as 11 prefixed safety examples, the maliciously fine-tuned LLMs will achieve similar safety performance as the original aligned models without harming the benign performance. Furthermore, we also present the effectiveness of our method in a more practical setting where the fine-tuning data consists of both FJAttack examples and the fine-tuning task data.

Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models

Neutralizing Backdoors through Information Conflicts for Large Language Models

CleanGen: Mitigating Backdoor Attacks for Generation Tasks in Large Language Models

Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges

AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment

BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models

Data Stealing Attacks against Large Language Models via Backdooring

MEGen: Generative Backdoor in Large Language Models via Model Editing

Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections

ASPIRER: Bypassing System Prompts With Permutation-based Backdoors in LLMs

Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment

Watch Out for Your Guidance on Generation! Exploring Conditional Backdoor Attacks against Large Language Models

Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers

CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization

Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems

Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor

Learning to Poison Large Language Models During Instruction Tuning

Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing

Flatness-aware Sequential Learning Generates Resilient Backdoors

Can Reinforcement Learning Unlock the Hidden Dangers in Aligned Large Language Models?

A Study of Backdoors in Instruction Fine-tuned Language Models