The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

Qianqian Xie,Weiguang Han,Yanzhao Lai,Min Peng,Jimin Huang

2023-04-28

Abstract:Recently, large language models (LLMs) like ChatGPT have demonstrated remarkable performance across a variety of natural language processing tasks. However, their effectiveness in the financial domain, specifically in predicting stock market movements, remains to be explored. In this paper, we conduct an extensive zero-shot analysis of ChatGPT's capabilities in multimodal stock movement prediction, on three tweets and historical stock price datasets. Our findings indicate that ChatGPT is a "Wall Street Neophyte" with limited success in predicting stock movements, as it underperforms not only state-of-the-art methods but also traditional methods like linear regression using price features. Despite the potential of Chain-of-Thought prompting strategies and the inclusion of tweets, ChatGPT's performance remains subpar. Furthermore, we observe limitations in its explainability and stability, suggesting the need for more specialized training or fine-tuning. This research provides insights into ChatGPT's capabilities and serves as a foundation for future work aimed at improving financial market analysis and prediction by leveraging social media sentiment and historical stock data.

Computation and Language,Machine Learning,Statistical Finance

What problem does this paper attempt to address?

The paper attempts to address the problem of evaluating the performance of ChatGPT in the task of multimodal stock price prediction. Specifically, the research focuses on the following aspects:

1. **Performance in Zero-Shot Setting**: Investigates whether ChatGPT can successfully predict stock price changes solely based on its strong language understanding capabilities without any specialized training.
2. **Impact of Prompt Strategies**: Explores the effect of different prompt strategies (including the Chain-of-Thought method) on the performance of ChatGPT.
3. **Role of Tweet Information**: Analyzes whether incorporating tweet information can enhance ChatGPT's predictive ability or if it introduces additional noise due to the long-tail distribution of stocks and the varying quality of tweets.
4. **Limitations in Explainability**: Evaluates ChatGPT's ability to explain its prediction results and points out its limitations.

Through these research questions, the paper aims to comprehensively assess the applicability and potential of ChatGPT in the task of multimodal stock price prediction and provide a foundation for future research.