PatentGPT: A Large Language Model for Intellectual Property

Zilong Bai,Ruiji Zhang,Linqing Chen,Qijun Cai,Yuan Zhong,Cong Wang,Yan Fang,Jie Fang,Jing Sun,Weikuan Wang,Lizhi Zhou,Haoran Hua,Tian Qiu,Chaochao Wang,Cheng Sun,Jianping Lu,Yixin Wang,Yubin Xia,Meng Hu,Haowen Liu,Peng Xu,Licong Xu,Fu Bian,Xiaolong Gu,Lisha Zhang,Weilei Wang,Changyang Tu

2024-06-05

Abstract:In recent years, large language models(LLMs) have attracted significant attention due to their exceptional performance across a multitude of natural language process tasks, and have been widely applied in various fields. However, the application of large language models in the Intellectual Property (IP) domain is challenging due to the strong need for specialized knowledge, privacy protection, processing of extremely long text in this field. In this technical report, we present for the first time a low-cost, standardized procedure for training IP-oriented LLMs, meeting the unique requirements of the IP domain. Using this standard process, we have trained the PatentGPT series models based on open-source pretrained models. By evaluating them on the open-source IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4, indicating the effectiveness of the proposed training procedure and the expertise of the PatentGPT models in the IP domain. Remarkably, our model surpassed GPT-4 on the 2019 China Patent Agent Qualification Examination, scoring 65 and matching human expert levels. Additionally, the PatentGPT model, which utilizes the SMoE architecture, achieves performance comparable to that of GPT-4 in the IP domain and demonstrates a better cost-performance ratio on long-text tasks, potentially serving as an alternative to GPT-4 within the IP domain.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the challenges faced when applying large language models (LLMs) in the field of intellectual property (IP), including the need for specialized knowledge, privacy protection requirements, and the need to handle extremely long texts. To solve these issues, the authors propose a low-cost, standardized training method specifically optimized for large language models in the IP field. Using this method, they trained a series of models named PatentGPT and evaluated them on the IP-specific benchmark MOZIP, showing that these models outperform GPT-4 in terms of performance. Additionally, the PatentGPT models achieved results comparable to human experts in the 2019 China Patent Attorney Qualification Examination, further demonstrating their expertise and effectiveness in the IP field.

Specifically, the main contributions of the paper include:
1. **Proposing a low-cost, standardized training process**: This process effectively trains large language models suitable for the IP field, meeting the unique needs of this domain.
2. **Developing the PatentGPT series of models**: Based on open-source pre-trained models, these models are trained with domain-specific data and outperform existing general large language models on IP-related tasks.
3. **Establishing the IP field benchmark PatentBench**: This is a comprehensive evaluation benchmark covering multiple tasks such as patent drafting, classification, and summarization, aiming to more accurately assess the performance of large language models specialized in the IP field.
4. **Demonstrating the potential of the models in practical applications**: The PatentGPT models not only excel in benchmark tests but also achieve near-human expert levels in the actual patent attorney qualification exam, showing their great potential in real-world applications.

In summary, the paper provides important references and tools for the research of large language models in the IP field by proposing a new training method and evaluation benchmark, contributing to the advancement of this domain.