Abstract:Watermarking generative models consists of planting a statistical signal (watermark) in a model's output so that it can be later verified that the output was generated by the given model. A strong watermarking scheme satisfies the property that a computationally bounded attacker cannot erase the watermark without causing significant quality degradation. In this paper, we study the (im)possibility of strong watermarking schemes. We prove that, under well-specified and natural assumptions, strong watermarking is impossible to achieve. This holds even in the private detection algorithm setting, where the watermark insertion and detection algorithms share a secret key, unknown to the attacker. To prove this result, we introduce a generic efficient watermark attack; the attacker is not required to know the private key of the scheme or even which scheme is used. Our attack is based on two assumptions: (1) The attacker has access to a "quality oracle" that can evaluate whether a candidate output is a high-quality response to a prompt, and (2) The attacker has access to a "perturbation oracle" which can modify an output with a nontrivial probability of maintaining quality, and which induces an efficiently mixing random walk on high-quality outputs. We argue that both assumptions can be satisfied in practice by an attacker with weaker computational capabilities than the watermarked model itself, to which the attacker has only black-box access. Furthermore, our assumptions will likely only be easier to satisfy over time as models grow in capabilities and modalities. We demonstrate the feasibility of our attack by instantiating it to attack three existing watermarking schemes for large language models: Kirchenbauer et al. (2023), Kuditipudi et al. (2023), and Zhao et al. (2023). The same attack successfully removes the watermarks planted by all three schemes, with only minor quality degradation.

Scalable Universal Adversarial Watermark Defending Against Facial Forgery

Warfare:Breaking the Watermark Protection of AI-Generated Content

Robust Adversarial Watermark Defending Against GAN Synthesization Attack

CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes

A Novel Model Watermarking for Protecting Generative Adversarial Network

Dual Defense: Adversarial, Traceable, and Invisible Robust Watermarking Against Face Swapping

Watermark-embedded Adversarial Examples for Copyright Protection against Diffusion Models

Towards Photo-Realistic Visible Watermark Removal with Conditional Generative Adversarial Networks

Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models

Generative Watermarking Against Unauthorized Subject-Driven Image Synthesis

Reliable Model Watermarking: Defending Against Theft without Compromising on Evasion

Watermark Faker: Towards Forgery of Digital Image Watermarking

Robustness of Watermarking on Text-to-Image Diffusion Models

Are Watermarks Bugs for Deepfake Detectors? Rethinking Proactive Forensics

Generating Image Adversarial Examples by Embedding Digital Watermarks

Robust Image Watermarking Based on Generative Adversarial Network

Double Privacy Guard: Robust Traceable Adversarial Watermarking against Face Recognition

Robust Identity Perceptual Watermark Against Deepfake Face Swapping

Investigating Deep Watermark Security: An Adversarial Transferability Perspective

ARWGAN: Attention-Guided Robust Image Watermarking Model Based on GAN