Abstract:Studies conducted on financial market prediction lack a comprehensive feature set that can carry a broad range of contributing factors; therefore, leading to imprecise results. Furthermore, while cooperating with the most recent innovations in explainable AI, studies have not provided an illustrative summary of market-driving factors using this powerful tool. Therefore, in this study, we propose a novel feature matrix that holds a broad range of features including Twitter content and market historical data to perform a binary classification task of one step ahead prediction. The utilization of our proposed feature matrix not only leads to improved prediction accuracy when compared to existing feature representations, but also its combination with explainable AI allows us to introduce a fresh analysis approach regarding the importance of the market-driving factors included. Thanks to the Lime interpretation technique, our interpretation study shows that the volume of tweets is the most important factor included in our feature matrix that drives the market's movements.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is that the current prediction research in the financial market lacks a comprehensive feature set and cannot cover a wide range of contributing factors, resulting in less accurate prediction results. In addition, although the latest Explainable AI (XAI) technology has been introduced, existing research has failed to provide a detailed summary and explanation of market - driving factors. Specifically, the paper points out the following two main problems: 1. **Incomplete feature set**: Existing financial market prediction models usually contain only limited features, such as historical price data or social media sentiment data, without effectively combining the two. This limits the model's understanding of market dynamics and prediction accuracy. 2. **Lack of interpretability**: Although deep - learning models perform well in the application of the financial market, due to their black - box nature, it is difficult to explain the prediction results of the model, which makes financial analysts and traders have less trust in them. To solve these problems, the author proposes a new feature matrix, which not only contains Twitter content (such as tweet sentiment, the number of tweets, the number of likes, etc.), but also combines the historical data of market prices (such as opening price, highest price, lowest price, closing price, etc.). In this way, the author hopes: - To improve prediction accuracy; - To use XAI technology (such as the LIME method) to explain the influence of each feature on the prediction result and increase the transparency and credibility of the model. ### Core objectives of the paper The core objective of the paper is to improve the financial market prediction model by introducing a comprehensive feature matrix and combining XAI technology. Specifically: - **Improve prediction accuracy**: By integrating a wider range of features, including social media content and market price data, enhance the prediction ability of the model. - **Enhance interpretability**: Use interpretation methods such as LIME to reveal which features have the greatest impact on market fluctuations and help understand market dynamics. ### Main innovation points 1. **Comprehensive feature matrix**: Contain Twitter content and market price data simultaneously to form a more comprehensive feature set. 2. **Interpretive analysis**: Use XAI technologies such as LIME to explain the contribution of each feature to the prediction result and provide a detailed analysis of market - driving factors. 3. **Dynamic dependence analysis**: For each prediction instance, provide an explanation of the influence of features on the result and show the dynamic changes in the market. Through these innovations, the paper aims to build a more accurate and transparent financial market prediction model to meet the needs of financial analysts and traders.

Stock Market Dynamics Through Deep Learning Context

Stock Market Prediction and Performance Analysis Using Machine Learning Algorithms

Media Moments and Corporate Connections: A Deep Learning Approach to Stock Movement Classification

Stock Price Movement Prediction from Financial News with Deep Learning and Knowledge Graph Embedding

Enhancing Stock Market Trend Prediction Using Explainable Artificial Intelligence and Multi-source Data

Stock Price Prediction using Multi-Faceted Information based on Deep Recurrent Neural Networks

Big Data: Deep Learning for financial sentiment analysis

Deep learning in finance assessing twitter sentiment impact and prediction on stocks

Feature Learning for Stock Price Prediction Shows a Significant Role of Analyst Rating

Deep Neural Networks Applied to Stock Market Sentiment Analysis

Deep Learning-Based Stock Market Prediction and Investment Model for Financial Management

Deep learning for Stock Market Prediction

Forecasting stock market returns using deep learning and time series techniques: a comparative and empirical study using technical indicators

Predicting Financial Market Trends using Time Series Analysis and Natural Language Processing

The Battle of Information Representations: Comparing Sentiment and Semantic Features for Forecasting Market Trends

Investigating Deep Stock Market Forecasting with Sentiment Analysis

Stock Price Predictions with LSTM Neural Networks and Twitter Sentiment

Deep Stock Ranker: A LSTM Neural Network Model for Stock Selection.

Incorporating Expert-Based Investment Opinion Signals in Stock Prediction: A Deep Learning Framework.

A stock price prediction method based on deep learning technology

Harvesting social media sentiment analysis to enhance stock market prediction using deep learning