Abstract:Labeling data is essential for training text classifiers but is often difficult to accomplish accurately, especially for complex and abstract concepts. Seeking an improved method, this paper employs a novel approach using a generative language model (GPT-4) to produce labels and rationales for large-scale text analysis. We apply this approach to the task of discovering public value expressions in US AI patents. We collect a database comprising 154,934 patent documents using an advanced Boolean query submitted to InnovationQ+. The results are merged with full patent text from the USPTO, resulting in 5.4 million sentences. We design a framework for identifying and labeling public value expressions in these AI patent sentences. A prompt for GPT-4 is developed which includes definitions, guidelines, examples, and rationales for text classification. We evaluate the quality of the labels and rationales produced by GPT-4 using BLEU scores and topic modeling and find that they are accurate, diverse, and faithful. These rationales also serve as a chain-of-thought for the model, a transparent mechanism for human verification, and support for human annotators to overcome cognitive limitations. We conclude that GPT-4 achieved a high-level of recognition of public value theory from our framework, which it also uses to discover unseen public value expressions. We use the labels produced by GPT-4 to train BERT-based classifiers and predict sentences on the entire database, achieving high F1 scores for the 3-class (0.85) and 2-class classification (0.91) tasks. We discuss the implications of our approach for conducting large-scale text analyses with complex and abstract concepts and suggest that, with careful framework design and interactive human oversight, generative language models can offer significant advantages in quality and in reduced time and costs for producing labels and rationales.

Predictive Patentomics: Forecasting Innovation Success and Valuation with ChatGPT

On Predictive Patent Valuation: Forecasting Patent Citations and Their Types.

Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models

ChatGPT, Generative AI, and Investment Advisory

Towards a Predictive Patent Analytics and Evaluation Platform

ChatGPT Can Predict the Future when it Tells Stories Set in the Future About the Past

ChatGPT-based Investment Portfolio Selection

Forecasting Emerging Technologies Using Data Augmentation and Deep Learning

From Transcripts to Insights: Uncovering Corporate Risks Using Generative AI

PatentGPT: A Large Language Model for Patent Drafting Using Knowledge-based Fine-tuning Method

Can Base ChatGPT be Used for Forecasting without Additional Optimization?

Prediction of Emerging Technologies Based on Analysis of the U.S. Patent Citation Network

Research on the Generation of Patented Technology Points in New Energy Based on Deep Learning

StockGPT: A GenAI Model for Stock Prediction and Trading

Large-Scale Text Analysis Using Generative Language Models: A Case Study in Discovering Public Value Expressions in AI Patents

Patent Value Analysis Using Deep Learning Models—The Case of IoT Technology Mining for the Manufacturing Industry

Marketable value estimation of patents using ensemble learning methodology: Focusing on U.S. patents for the electricity sector

Discovering the realistic paths towards the realization of patent valuation from technical perspectives: defense, implementation or transfer

Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model

Discovery of factors influencing patent value based on machine learning in patents in the field of nanotechnology

Which will be your firm's next technology? Comparison between machine learning and network-based algorithms