Attributing Image Generative Models using Latent Fingerprints

Guangyu Nie,Changhoon Kim,Yezhou Yang,Yi Ren

2023-05-27

Abstract:Generative models have enabled the creation of contents that are indistinguishable from those taken from nature. Open-source development of such models raised concerns about the risks of their misuse for malicious purposes. One potential risk mitigation strategy is to attribute generative models via fingerprinting. Current fingerprinting methods exhibit a significant tradeoff between robust attribution accuracy and generation quality while lacking design principles to improve this tradeoff. This paper investigates the use of latent semantic dimensions as fingerprints, from where we can analyze the effects of design variables, including the choice of fingerprinting dimensions, strength, and capacity, on the accuracy-quality tradeoff. Compared with previous SOTA, our method requires minimum computation and is more applicable to large-scale models. We use StyleGAN2 and the latent diffusion model to demonstrate the efficacy of our method.

Computer Vision and Pattern Recognition,Cryptography and Security,Machine Learning

What problem does this paper attempt to address?

The paper primarily addresses the issue of responsibility attribution when generative models are maliciously used, specifically how to identify the specific model source of generated content through fingerprinting techniques. Specifically, the paper attempts to solve the following key problems:

1. **Balancing attribution accuracy and generation quality**: Existing fingerprinting methods often lead to a decline in the quality of generated images while ensuring high attribution accuracy. The paper proposes a new method that uses latent semantic dimensions as fingerprints to analyze the impact of design variables (such as the choice, strength, and capacity of fingerprint dimensions) on the accuracy-quality trade-off.

2. **Reducing computational cost**: Compared to previous methods that require additional training or complex computational processes, the proposed method requires minimal computation and is more suitable for large-scale models.

3. **Enhancing robustness**: Existing shallow fingerprinting methods perform poorly against attempts to remove attribution through image post-processing. The proposed method uses subtle but semantically meaningful changes in the latent space as fingerprints to improve robustness against image post-processing.

4. **Improving applicability and scalability**: The proposed method is not only applicable to models like StyleGAN2 but also to Latent Diffusion Models (LDM), demonstrating its broad applicability and scalability to large models.

In summary, this paper aims to improve the responsibility attribution mechanism of generative models by introducing a novel latent fingerprinting strategy that effectively balances attribution accuracy and generation quality, with lower computational costs and stronger robustness.