Abstract:Diffusion models have rapidly become a vital part of deep generative architectures, given today's increasing demands. Obtaining large, high-performance diffusion models demands significant resources, highlighting their importance as intellectual property worth protecting. However, existing watermarking techniques for ownership verification are insufficient when applied to diffusion models. Very recent research in watermarking diffusion models either exposes watermarks during task generation, which harms the imperceptibility, or is developed for conditional diffusion models that require prompts to trigger the watermark. This paper introduces WDM, a novel watermarking solution for diffusion models without imprinting the watermark during task generation. It involves training a model to concurrently learn a Watermark Diffusion Process (WDP) for embedding watermarks alongside the standard diffusion process for task generation. We provide a detailed theoretical analysis of WDP training and sampling, relating it to a shifted Gaussian diffusion process via the same reverse noise. Extensive experiments are conducted to validate the effectiveness and robustness of our approach in various trigger and watermark data configurations.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is **intellectual property protection of diffusion models**. Specifically, with the increasingly wide application of diffusion models in generating high - quality samples, as important digital assets, the intellectual property protection of these models has become particularly important. However, existing watermarking techniques have deficiencies when applied to diffusion models. For example, the watermark is exposed during the task generation process, which impairs the concealment of the watermark, or can only be applied to conditional diffusion models that require prompt words to trigger the watermark. To this end, the paper proposes a new watermarking scheme - **WDM (Watermark Diffusion Models)**, which embeds watermarks by introducing the **Watermark Diffusion Process (WDP)** without explicitly adding watermarks during the task generation process. The core idea of WDM is to simultaneously learn a standard diffusion process for task generation and a WDP for embedding watermarks when training or fine - tuning the model. In this way, even during the task generation process, the watermark will not be exposed, thus ensuring the concealment of the watermark and the performance of the model. ### Main contributions 1. **Proposed a new watermarking scheme**: Embed watermarks through WDP without interfering with the task generation process. 2. **Theoretical basis and analysis**: Analyzed in detail the training and sampling processes of WDP and related it to the modified Gaussian diffusion process. 3. **Complete framework**: Proposed a complete framework including three stages: watermark embedding, extraction and verification. 4. **Experimental verification**: Verified the effectiveness and robustness of the method through extensive experiments and can resist multiple types of attacks. ### Technical details - **Watermark embedding**: Embed watermark data by simultaneously learning the standard diffusion process and WDP. - **Watermark extraction**: Extract watermark data from the target model by using the reverse noise shared by WDP. - **Watermark verification**: Verify the existence of the watermark through hypothesis testing and similarity comparison. ### Experimental results - **Model fidelity**: After embedding the watermark, the performance of the model in task generation hardly decreases. - **Watermark fidelity**: The embedded watermark can be successfully extracted and its existence is verified through hypothesis testing. - **Robustness**: WDM has strong robustness against attacks such as model compression, weight perturbation and model fine - tuning. In conclusion, this paper proposes an effective watermarking scheme that can protect the intellectual property of diffusion models without sacrificing the performance of the models.

Intellectual Property Protection of Diffusion Models via the Watermark Diffusion Process

Embedding Watermarks in Diffusion Process for Model Intellectual Property Protection

A Watermark-Conditioned Diffusion Model for IP Protection

Warfare:Breaking the Watermark Protection of AI-Generated Content

Watermarking Diffusion Model

A Recipe for Watermarking Diffusion Models

Watermarking for Stable Diffusion Models

DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models

Exploiting Watermark-Based Defense Mechanisms in Text-to-Image Diffusion Models for Unauthorized Data Usage

Invisible Watermarking for Audio Generation Diffusion Models

Protecting Copyright of Stable Diffusion Models from Ambiguity Attacks

DiffuseTrace: A Transparent and Flexible Watermarking Scheme for Latent Diffusion Model

WMAdapter: Adding WaterMark Control to Latent Diffusion Models

Robustness of Watermarking on Text-to-Image Diffusion Models

ROBIN: Robust and Invisible Watermarks for Diffusion Models with Adversarial Optimization

Shallow Diffuse: Robust and Invisible Watermarking through Low-Dimensional Subspaces in Diffusion Models

Hey That's Mine Imperceptible Watermarks are Preserved in Diffusion Generated Outputs

Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models

SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models

Reliable Model Watermarking: Defending Against Theft without Compromising on Evasion

WaterDiff: Perceptual Image Watermarks Via Diffusion Model