ProFLingo: A Fingerprinting-based Intellectual Property Protection Scheme for Large Language Models

Heng Jin,Chaoyu Zhang,Shanghao Shi,Wenjing Lou,Y. Thomas Hou
2024-09-11
Abstract:Large language models (LLMs) have attracted significant attention in recent years. Due to their "Large" nature, training LLMs from scratch consumes immense computational resources. Since several major players in the artificial intelligence (AI) field have open-sourced their original LLMs, an increasing number of individuals and smaller companies are able to build derivative LLMs based on these open-sourced models at much lower costs. However, this practice opens up possibilities for unauthorized use or reproduction that may not comply with licensing agreements, and fine-tuning can change the model's behavior, thus complicating the determination of model ownership. Current intellectual property (IP) protection schemes for LLMs are either designed for white-box settings or require additional modifications to the original model, which restricts their use in real-world settings. In this paper, we propose ProFLingo, a black-box fingerprinting-based IP protection scheme for LLMs. ProFLingo generates queries that elicit specific responses from an original model, thereby establishing unique fingerprints. Our scheme assesses the effectiveness of these queries on a suspect model to determine whether it has been derived from the original model. ProFLingo offers a non-invasive approach, which neither requires knowledge of the suspect model nor modifications to the base model or its training process. To the best of our knowledge, our method represents the first black-box fingerprinting technique for IP protection for LLMs. Our source code and generated queries are available at: <a class="link-external link-https" href="https://github.com/hengvt/ProFLingo" rel="external noopener nofollow">this https URL</a>.
Cryptography and Security,Machine Learning
What problem does this paper attempt to address?
### Problems Addressed by the Paper The paper aims to address the issue of intellectual property protection for large language models (LLMs). Specifically, due to the high training costs of large language models, many individuals and small companies choose to fine-tune open-source models to create derivative models. However, this practice may lead to unauthorized use or replication, violating licensing agreements, and the fine-tuning process can alter the model's behavior, making it complex to determine model ownership. Current intellectual property protection schemes are either designed for white-box settings or require modifications to the original model, limiting their applicability in real-world scenarios. The paper proposes ProFLingo, a black-box intellectual property protection scheme based on fingerprinting, suitable for large language models. ProFLingo establishes a unique fingerprint by generating queries that elicit specific responses from the original model and evaluates the effectiveness of these queries on the suspected model to determine if it is derived from the original model. This method does not require knowledge of the specific details of the suspected model, nor does it require any modifications to the base model or its training process. To the best of the authors' knowledge, this is the first black-box fingerprinting technique for large language models.