Abstract:To trace the copyright of deep neural networks, an owner can embed its identity information into its model as a watermark. The capacity of the watermark quantify the maximal volume of information that can be verified from the watermarked model. Current studies on capacity focus on the ownership verification accuracy under ordinary removal attacks and fail to capture the relationship between robustness and fidelity. This paper studies the capacity of deep neural network watermarks from an information theoretical perspective. We propose a new definition of deep neural network watermark capacity analogous to channel capacity, analyze its properties, and design an algorithm that yields a tight estimation of its upper bound under adversarial overwriting. We also propose a universal non-invasive method to secure the transmission of the identity message beyond capacity by multiple rounds of ownership verification. Our observations provide evidence for neural network owners and defenders that are curious about the tradeoff between the integrity of their ownership and the performance degradation of their products.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the information capacity problem of deep neural network (DNN) watermarking. Specifically, the paper focuses on how to maximize the amount of embedded watermark information while protecting the model's copyright, and maintain robustness and fidelity in the face of adversarial attacks. ### Main contributions of the paper 1. **Defined the information - theory - based DNN watermark capacity**: - The authors proposed a brand - new definition, analogizing the capacity of DNN watermarking to channel capacity and analyzing its properties. - The capacity is defined as the maximum amount of information that can be correctly transmitted when the model performance degradation does not exceed a certain threshold. 2. **Designed an algorithm for estimating the upper bound of capacity**: - This algorithm can provide a tight upper - bound estimate of capacity under adversarial overlay attacks. - By introducing adversarial overlay attacks, directly tampering with the watermark while maintaining the model performance, a more stringent capacity estimate is obtained. 3. **Proposed a non - invasive multi - round ownership verification method**: - This method can improve the accuracy of identity information transmission through multiple verifications, exceeding the capacity limit. - The method is applicable to any DNN watermarking scheme and does not require modification of the original model structure. ### Mathematical formulas - **Definition of capacity**: \[ C(\delta, L)=\min_{\theta}\left\{\max_{p(m)}I(m; \hat{m})\right\} \] where \(\hat{m}=\text{Verify}(M_{WM}+\theta, K)\) and satisfies \(E(M_{WM}+\theta)\geq E(M_{WM})-\delta\). - **Upper bound of BER**: \[ C(\delta, L)\leq L\cdot(1 - H(\epsilon_\delta)) \] where \(H(x)=-x\cdot\log_2x-(1 - x)\cdot\log_2(1 - x)\) is the binary entropy function, and \(\epsilon_\delta=\max_{\theta\in\Theta(\delta)}\left\{\epsilon(\theta)\right\}\) is the maximum bit error rate. - **Minimum identity message length**: \[ \tilde{L}=\min_L\left\{L:\left(F(L)+\min_\delta\left\{\delta:C(\delta, L)\leq J\right\}\right)\geq\Delta\right\} \] ### Experimental results Through experiments on several representative DNN watermarking schemes, the paper verified the effectiveness of the proposed theories and methods. The results show that the MTLSign scheme has the largest capacity, but it is not necessarily optimal in terms of performance degradation. Therefore, a good watermarking scheme should strike a balance between capacity and fidelity. ### Conclusion This paper, by re - examining the information capacity of DNN watermarking, provides a new information - theory - based definition and designs corresponding algorithms and methods to estimate and enhance the capacity. These studies provide important references for DNN model owners and defenders, helping them protect intellectual property rights while ensuring that the model's performance is not overly affected.

Revisiting the Information Capacity of Neural Network Watermarks: Upper Bound Estimation and Beyond

Leveraging Unlabeled Data for Watermark Removal of Deep Neural Networks

Ownership Verification of DNN Architectures Via Hardware Cache Side Channels.

OVLA: Neural Network Ownership Verification using Latent Watermarks

Knowledge-Free Black-Box Watermark and Ownership Proof for Image Classification Neural Networks

Watermarking Neural Networks with Watermarked Images

Deep Model Intellectual Property Protection Via Deep Watermarking

On the Weaknesses of Backdoor-based Model Watermarking: An Information-theoretic Perspective

Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication

Persistent and Unforgeable Watermarks for Deep Neural Networks.

Towards Robust Model Watermark Via Reducing Parametric Vulnerability

Watermarking Deep Neural Networks With Greedy Residuals

Regulating Ownership Verification for Deep Neural Networks: Scenarios, Protocols, and Prospects

On Function-Coupled Watermarks for Deep Neural Networks

A Robustness-Assured White-Box Watermark in Neural Networks

Watermarking in Deep Neural Networks Via Error Back-propagation

Protecting the Intellectual Property of Deep Neural Networks with Watermarking: The Frequency Domain Approach

Deep neural networks watermark via universal deep hiding and metric learning

Rethinking White-BoxWatermarks on Deep Learning Models under Neural Structural Obfuscation

Certified Neural Network Watermarks with Randomized Smoothing

Protecting the Intellectual Properties of Deep Neural Networks with an Additional Class and Steganographic Images