Abstract:In the rapidly evolving field of financial sentiment analysis, the efficiency and accuracy of predictive models are critical due to their significant impact on financial markets. Transformer based models like BERT and large language models (LLMs) like GPT-4, have advanced NLP tasks considerably. Despite their advantages, BERT-based models face challenges with computational intensity in edge computing environments, and the substantial size and compute requirements of LLMs limit their practical deployment. This study proposes leveraging the generative capabilities of LLMs, such as GPT-4 Omni, to create synthetic, domain-specific training data. This approach addresses the challenge of data scarcity and enhances the performance of smaller models by making them competitive with their larger counterparts. The research specifically aims to enhance FinBERT, a BERT model fine-tuned for financial sentiment analysis, and develop TinyFinBERT, a compact transformer model, through a structured, two-tiered knowledge distillation strategy. Using data augmented by GPT-4 Omni, which involves generating new training examples and transforming existing data, we significantly improved the accuracy of FinBERT, preparing it to serve as a teacher model. This enhanced FinBERT then distilled knowledge to TinyFinBERT, employing both GPT-4 Omni and GPT-3.5 Turbo augmented data. The distillation strategy incorporated both logit and intermediate layer distillation. The training and evaluation of TinyFinBERT utilized the PhraseBank dataset and the FiQA 2018 Task1 dataset, achieving performance comparable to FinBERT while being substantially smaller and more efficient. This research demonstrates how LLMs can effectively contribute to the advancement of financial sentiment analysis by enhancing the capabilities of smaller, more efficient models through innovative data augmentation and distillation techniques.

Distilled ChatGPT Topic & Sentiment Modeling with Applications in Finance

Transforming Sentiment Analysis in the Financial Domain with ChatGPT

Using ChatGPT to Predict the Equity Markets - PSL Week Slides

Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance

Predict financial text sentiment: an empirical examination

The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

ChatGPT for (Finance) Research: The Bananarama Conjecture

ChatGPT-based Investment Portfolio Selection

Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing

Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis

Applying QNLP to sentiment analysis in finance

ChatGPT-style Artificial Intelligence for Financial Applications and Risk Response

How Much Does ChatGPT Know about Finance?

Sentiment Analysis of Citations in Scientific Articles Using ChatGPT: Identifying Potential Biases and Conflicts of Interest

Enhancing TinyBERT for Financial Sentiment Analysis Using GPT-Augmented FinBERT Distillation

ChatGPT in finance: Applications, challenges, and solutions

Investor Sentiment Analysis of Financial Texts Based on GPT and RoBERTa

Using Knowledge Distillation to improve interpretable models in a retail banking context

Potential of ChatGPT in predicting stock market trends based on Twitter Sentiment Analysis

Evaluation of Sentiment Analysis in Finance: From Lexicons to Transformers

Harmonizing Macro-Financial Factors and Twitter Sentiment Analysis in Forecasting Stock Market Trends