Abstract:Watermark algorithms for large language models (LLMs) have achieved extremely high accuracy in detecting text generated by LLMs. Such algorithms typically involve adding extra watermark logits to the LLM's logits at each generation step. However, prior algorithms face a trade-off between attack robustness and security robustness. This is because the watermark logits for a token are determined by a certain number of preceding tokens; a small number leads to low security robustness, while a large number results in insufficient attack robustness. In this work, we propose a semantic invariant watermarking method for LLMs that provides both attack robustness and security robustness. The watermark logits in our work are determined by the semantics of all preceding tokens. Specifically, we utilize another embedding LLM to generate semantic embeddings for all preceding tokens, and then these semantic embeddings are transformed into the watermark logits through our trained watermark model. Subsequent analyses and experiments demonstrated the attack robustness of our method in semantically invariant settings: synonym substitution and text paraphrasing settings. Finally, we also show that our watermark possesses adequate security robustness. Our code and data are available at \href{

What problem does this paper attempt to address?

This paper aims to solve the trade - off problem between attack robustness and security robustness in watermarking algorithms for text generation by large - language models (LLMs). Specifically, current watermarking algorithms usually achieve high - precision detection by adding additional watermark logits to the logits of the LLM at each generation step. However, these algorithms face a dilemma: the number of watermark logits depends on the number of previous tokens. If this number is small, the security is low; if it is large, the attack robustness is insufficient. To solve this problem, the author proposes a semantically - invariant watermarking method that can provide both attack robustness and security robustness simultaneously. The core of this method is that the generation of watermark logits no longer depends on the specific identities of the previous few tokens, but on the semantics of all previous tokens. In terms of specific implementation, the author uses an auxiliary embedded LLM to generate semantic embeddings of all previous tokens, and then converts these semantic embeddings into watermark logits through a trained watermark model. In this way, even if semantically - invariant modifications such as synonym replacement or text restatement are made to the text, the watermark logits will not be significantly affected, thereby improving attack robustness. At the same time, since the generation of watermark logits depends on complex semantic information rather than simple token identities, it also increases the difficulty of cracking the watermark rules and improves security robustness. The paper verifies the robustness of this method under various semantically - invariant attacks through experiments and shows its superior performance in terms of security robustness. Overall, the method proposed in this paper effectively solves the trade - off problem between attack robustness and security robustness in existing watermarking algorithms.

A Semantic Invariant Robust Watermark for Large Language Models

Adaptive Text Watermark for Large Language Models

A Robust Semantics-based Watermark for Large Language Model against Paraphrasing

Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models

Signal Watermark on Large Language Models

REMARK-LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models

Unbiased Watermark for Large Language Models

Provably Robust Watermarks for Open-Source Language Models

Advancing Beyond Identification: Multi-bit Watermark for Large Language Models

On the Reliability of Watermarks for Large Language Models

Universally Optimal Watermarking Schemes for LLMs: from Theory to Practice

Necessary and Sufficient Watermark for Large Language Models

Learnable Linguistic Watermarks for Tracing Model Extraction Attacks on Large Language Models

PostMark: A Robust Blackbox Watermark for Large Language Models

A Watermark for Low-entropy and Unbiased Generation in Large Language Models

Large Language Model Watermark Stealing With Mixed Integer Programming

A Certified Robust Watermark For Large Language Models

Turning Your Strength into Watermark: Watermarking Large Language Model via Knowledge Injection

Topic-Based Watermarks for LLM-Generated Text

Baselines for Identifying Watermarked Large Language Models

A Watermark for Large Language Models