Abstract:Consistency models are a new class of models that generate images by directly mapping noise to data, allowing for one-step generation and significantly accelerating the sampling process. However, their robustness against adversarial attacks has not yet been thoroughly investigated. In this work, we conduct the first study on the vulnerability of consistency models to backdoor attacks. While previous research has explored backdoor attacks on diffusion models, these studies have primarily focused on conventional diffusion models, employing a customized backdoor training process and objective, whereas consistency models have distinct training processes and objectives. Our proposed framework demonstrates the vulnerability of consistency models to backdoor attacks. During image generation, poisoned consistency models produce images with a Fréchet Inception Distance (FID) comparable to that of a clean model when sampling from Gaussian noise. However, once the trigger is activated, they generate backdoor target images. We explore various trigger and target configurations to evaluate the vulnerability of consistency models, including the use of random noise as a trigger. This type of trigger is less conspicuous and aligns well with the sampling process of consistency models. Across all configurations, our framework successfully compromises the consistency models while maintaining high utility and specificity.

What problem does this paper attempt to address?

The problem this paper attempts to address is: **the vulnerability of Consistency Models to Backdoor Attacks**. Specifically, the paper points out that although consistency models have significant advantages in image generation, such as achieving one-step generation by directly mapping noise to data, thereby significantly accelerating the sampling process, their robustness to adversarial attacks (especially backdoor attacks) has not been fully studied. Therefore, the authors explore for the first time the vulnerability of consistency models to backdoor attacks in this work. ### Main Research Objectives: 1. **Explore how to perform backdoor attacks on consistency models**: The authors propose a framework that demonstrates how to inject backdoor triggers and target images during training, enabling the consistency model to generate malicious images when the trigger is activated. 2. **Evaluate the performance of consistency models under different backdoor settings**: The authors experimentally verify the vulnerability of consistency models under different trigger and target configurations, including using random noise as a trigger, which is more covert during the sampling process of consistency models. ### Specific Issues: - **How to successfully inject a backdoor without degrading the normal image generation quality of the model**: That is, the model should be able to generate high-quality images normally when the backdoor is not triggered; once the backdoor is triggered, the model should generate specific target images. - **How to design covert and effective triggers**: The authors propose using Gaussian noise as a trigger, which is not easily detected during the generation process of consistency models. Through this research, the authors hope to reveal the potential security risks of consistency models and provide references for future research and applications.

How to Backdoor Consistency Models?

Palette: Physically-Realizable Backdoor Attacks Against Video Recognition Models

Invisible Backdoor Attacks on Diffusion Models

How to Backdoor Diffusion Models?

Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning

Backdoor Attack with Mode Mixture Latent Modification

Invisible Intruders: Label-Consistent Backdoor Attack Using Re-parameterized Noise Trigger

How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

Truncated Consistency Models

TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors

Adaptive Backdoor Attack Against Deep Neural Networks

Backdooring Bias into Text-to-Image Models

Attack as Defense: Run-time Backdoor Implantation for Image Content Protection

Stand-in Backdoor: A Stealthy and Powerful Backdoor Attack

Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers

Countering Backdoor Attacks in Image Recognition: A Survey and Evaluation of Mitigation Strategies

Enhanced Coalescence Backdoor Attack Against DNN Based on Pixel Gradient

Clean-Label Backdoor Attacks on Video Recognition Models

A stealthy and robust backdoor attack via frequency domain transform

An Effective and Resilient Backdoor Attack Framework against Deep Neural Networks and Vision Transformers