Mist: Towards Improved Adversarial Examples for Diffusion Models

Chumeng Liang,Xiaoyu Wu

2023-05-22

Abstract:Diffusion Models (DMs) have empowered great success in artificial-intelligence-generated content, especially in artwork creation, yet raising new concerns in intellectual properties and copyright. For example, infringers can make profits by imitating non-authorized human-created paintings with DMs. Recent researches suggest that various adversarial examples for diffusion models can be effective tools against these copyright infringements. However, current adversarial examples show weakness in transferability over different painting-imitating methods and robustness under straightforward adversarial defense, for example, noise purification. We surprisingly find that the transferability of adversarial examples can be significantly enhanced by exploiting a fused and modified adversarial loss term under consistent parameters. In this work, we comprehensively evaluate the cross-method transferability of adversarial examples. The experimental observation shows that our method generates more transferable adversarial examples with even stronger robustness against the simple adversarial defense.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

This paper aims to solve the copyright infringement issues caused by Diffusion Models (DMs) when generating artworks. Specifically, DMs can be conveniently used to imitate and transform artistic styles, which enables unauthorized artworks to be used to generate new works, thus potentially infringing on the original authors' copyrights. To prevent this situation, researchers have proposed adversarial examples as a means of protection. However, existing adversarial examples are weak in cross - method transferability and robustness against simple adversarial defenses. For example, some adversarial examples are effective in image - to - image generation but ineffective in textual inversion. To solve these problems, this paper proposes a new method to generate more transferable and robust adversarial examples. By fusing and modifying the adversarial loss terms and optimizing under consistent parameters, the authors of the paper find that the transferability of adversarial examples can be significantly enhanced. In addition, the paper also explores how the selection of different target images affects the performance of adversarial examples and finds that target images with high contrast and sharp edges can produce more effective adversarial examples. Overall, this paper improves the transferability of adversarial examples among different painting imitation methods and their resistance to simple adversarial defenses by improving the method of generating adversarial examples, thus providing an effective technical means to protect artists' copyrights.

Mist: Towards Improved Adversarial Examples for Diffusion Models

Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples

Improving Adversarial Transferability by Stable Diffusion

AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Models

Exploring Adversarial Attacks against Latent Diffusion Model from the Perspective of Adversarial Transferability

StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model

Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Models

Understanding and Enhancing the Transferability of Adversarial Examples

Adversarial Examples are Misaligned in Diffusion Model Manifolds

Robust Diffusion Models for Adversarial Purification

DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing

Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models

Watermark-embedded Adversarial Examples for Copyright Protection against Diffusion Models

Improving Transferability of Adversarial Examples With Input Diversity

Unveiling Universal Forensics of Diffusion Models with Adversarial Perturbations

The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipeline

Diffusion Models for Imperceptible and Transferable Adversarial Attack

Boosting the Targeted Transferability of Adversarial Examples via Salient Region & Weighted Feature Drop

Toward Transferable Attack via Adversarial Diffusion in Face Recognition

Targeted Attack Improves Protection against Unauthorized Diffusion Customization

Adversarial defense based on distribution transfer