Abstract:AI-Generated Content (AIGC) is gaining great popularity, with many emerging commercial services and applications. These services leverage advanced generative models, such as latent diffusion models and large language models, to generate creative content (e.g., realistic images and fluent sentences) for users. The usage of such generated content needs to be highly regulated, as the service providers need to ensure the users do not violate the usage policies (e.g., abuse for commercialization, generating and distributing unsafe content). A promising solution to achieve this goal is watermarking, which adds unique and imperceptible watermarks on the content for service verification and attribution. Numerous watermarking approaches have been proposed recently. However, in this paper, we show that an adversary can easily break these watermarking mechanisms. Specifically, we consider two possible attacks. (1) Watermark removal: the adversary can easily erase the embedded watermark from the generated content and then use it freely bypassing the regulation of the service provider. (2) Watermark forging: the adversary can create illegal content with forged watermarks from another user, causing the service provider to make wrong attributions. We propose Warfare, a unified methodology to achieve both attacks in a holistic way. The key idea is to leverage a pre-trained diffusion model for content processing and a generative adversarial network for watermark removal or forging. We evaluate Warfare on different datasets and embedding setups. The results prove that it can achieve high success rates while maintaining the quality of the generated content. Compared to existing diffusion model-based attacks, Warfare is 5,050~11,000x faster.

Protecting Intellectual Property of Large Language Model-Based Code Generation APIs Via Watermarks

Warfare:Breaking the Watermark Protection of AI-Generated Content

Protecting Intellectual Property of Language Generation APIs with Lexical Watermark

Protecting Language Generation Models via Invisible Watermarking

A novel watermarking framework for intellectual property protection of NLG APIs.

Protecting Your NLG Models with Semantic and Robust Watermarks

Learnable Linguistic Watermarks for Tracing Model Extraction Attacks on Large Language Models

Watermarking Large Language Models and the Generated Content: Opportunities and Challenges

CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code

ModelShield: Adaptive and Robust Watermark against Model Extraction Attack

Turning Your Strength into Watermark: Watermarking Large Language Model via Knowledge Injection

A Watermark for Large Language Models

An Effective Framework for Intellectual Property Protection of NLG Models

Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data?

Large Language Model Watermark Stealing With Mixed Integer Programming

Watermark Stealing in Large Language Models

WAPITI: A Watermark for Finetuned Open-Source LLMs

Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark

Provably Robust Watermarks for Open-Source Language Models

Watermarking Pre-trained Language Models with Backdooring

Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning