Abstract:Recently, Diffusion Models (DMs) boost a wave in AI for Art yet raise new copyright concerns, where infringers benefit from using unauthorized paintings to train DMs to generate novel paintings in a similar style. To address these emerging copyright violations, in this paper, we are the first to explore and propose to utilize adversarial examples for DMs to protect human-created artworks. Specifically, we first build a theoretical framework to define and evaluate the adversarial examples for DMs. Then, based on this framework, we design a novel algorithm, named AdvDM, which exploits a Monte-Carlo estimation of adversarial examples for DMs by optimizing upon different latent variables sampled from the reverse process of DMs. Extensive experiments show that the generated adversarial examples can effectively hinder DMs from extracting their features. Therefore, our method can be a powerful tool for human artists to protect their copyright against infringers equipped with DM-based AI-for-Art applications. The code of our method is available on GitHub: <a class="link-external link-https" href="https://github.com/mist-project/mist.git" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to use adversarial examples to prevent copyright infringement in artistic creation by Diffusion Models (DMs). Specifically, in recent years, diffusion models have made remarkable progress in AI - generated art, but at the same time, they have also given rise to new copyright issues. Some infringers use unauthorized artworks to train diffusion models to generate new paintings with similar styles. In order to address these emerging copyright infringements, this paper, for the first time, explores and proposes a method of using adversarial examples to protect human - created artworks. ### Specific Problem Description 1. **Copyright Infringement Problem**: Diffusion models can learn from a small number of samples and imitate the unique styles of artists, which has led to some copyright infringement behaviors. For example, the textual inversion technique can quickly learn and imitate the style of a specific artist using a small number of samples, enabling infringers to illegally use these artworks to train models. 2. **Lack of Effective Protection Means**: Although artists have the right to declare that their works are prohibited from being used to train AI art applications, currently, there is a lack of effective technical means to prevent or track such illegal use, resulting in low infringement costs and difficult evidence - collection. ### Solutions 1. **Construct a Theoretical Framework**: This paper first constructs a theoretical framework to define and evaluate adversarial examples against diffusion models. These adversarial examples add tiny perturbations to images, making it impossible for diffusion models to extract image features or imitate artistic styles. 2. **Design an Algorithm**: Based on the above - mentioned theoretical framework, this paper designs a new algorithm named AdvDM. This algorithm uses the Monte Carlo method to estimate the objective function of adversarial examples and generates adversarial examples by optimizing different latent variables. 3. **Experimental Verification**: Through extensive experiments, this paper verifies that the generated adversarial examples can effectively prevent diffusion models from extracting features and imitating styles from images. ### Main Contributions 1. **Construct a New Framework**: This paper, for the first time, systematically studies the definition and evaluation methods of adversarial examples against diffusion models. 2. **Propose a New Algorithm**: Proposes the AdvDM algorithm for generating adversarial examples against diffusion models. 3. **Experimental Verification**: Through experiments on multiple datasets, verifies the effectiveness of the method in this paper, especially in protecting images from being learned, imitated, and copied by diffusion models. ### Technical Details - **Objective Function**: Define the objective function of adversarial examples to minimize the probability that adversarial examples are recognized as real images. - **Algorithm Implementation**: Use the Monte Carlo method to estimate the objective function and generate adversarial examples by the gradient - ascent method. - **Evaluation Method**: Evaluate the effect of adversarial examples by the quality of generated images, especially the quality of conditionally - generated images. ### Experimental Results - **Text - to - Image Generation**: On the LSUN and WikiArt datasets, the generated adversarial examples significantly increase the FID value and decrease the Precision value, indicating that the adversarial examples effectively protect the image content from being extracted as generation conditions. - **Style Transfer**: On the WikiArt dataset, the generated adversarial examples make the style of conditionally - generated images significantly different from that of the input images, indicating that the adversarial examples can effectively prevent illegal style transfer. - **Image - to - Image Synthesis**: On the Pexels dataset, the generated adversarial examples make the conditionally - generated images unrealistic, further verifying the effectiveness of the adversarial examples. Through these methods and techniques, this paper provides artists with an effective tool to protect their works from illegal use by AI art applications.

Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples

Mist: Towards Improved Adversarial Examples for Diffusion Models

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Warfare:Breaking the Watermark Protection of AI-Generated Content

Adversarial Examples for Preventing Diffusion Models from Malicious Image Edition

Watermark-embedded Adversarial Examples for Copyright Protection against Diffusion Models

My Art My Choice: Adversarial Protection Against Unruly AI

The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipeline

DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models

Adversarial examples for models of code

Human-Producible Adversarial Examples

Simple Transparent Adversarial Examples

Generating Image Adversarial Examples by Embedding Digital Watermarks

Toward effective protection against diffusion based mimicry through score distillation

Dual Protection for Image Privacy and Copyright Via Traceable Adversarial Examples

Deceptive Diffusion: Generating Synthetic Adversarial Examples

Measuring the Success of Diffusion Models at Imitating Human Artists

Benign Adversarial Attack

Efficiently Constructing Adversarial Examples by Feature Watermarking.

AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Models

Unrestricted Adversarial Examples via Semantic Manipulation