Abstract:With the ubiquity of the Internet, platforms such as: Google, Wikipedia and the like can provide insights pertaining to firms’ financial performance as well as capture the collective interest of traders through search trends, number of web page visitors and/or financial news sentiment. Information emanating from these platforms can significantly affect, or be affected by, changes in the stock market. The overarching goal of this paper is to develop a financial expert system that incorporates these features to predict short term stock prices. Our expert system is comprised of two main modules: a knowledge base and an artificial intelligence (AI) platform. The “knowledge base” for our expert system captures: (a) historical stock prices; (b) several well-known technical indicators; (c) counts and sentiment scores of published news articles for a given stock; (d) trends in Google searches for the given stock ticker; and (e) number of unique visitors for pertinent Wikipedia pages. Once the data is collected, we use a structured approach for data preparation. Then, the AI platform trains four machine learning ensemble methods: (a) a neural network regression ensemble; (b) a support vector regression ensemble; (c) a boosted regression tree; and (d) a random forest regression. In the cross-validation phase, the AI platform picks the “best” ensemble for a given stock. To evaluate the efficacy of our expert system, we first present a case study based on the Citi Group stock ($C) with data collected from 01/01/2013 - 12/31/2016. We show the expert system can predict the 1-day ahead $C stock price with a mean absolute percent error (MAPE)  ≤  1.50% and the 1–10 day ahead with a MAPE  ≤  1.89%, which is better than the reported results in the literature. We show that the use of features extracted from online sources does not substitute the traditional financial metrics, but rather supplements them to improve upon the prediction performance of machine learning based methods. To highlight the utility and generalizability of our expert system, we predict the 1-day ahead price of 19 additional stocks from different industries, volatilities and growth patterns. We report an overall mean for the MAPE statistic of 1.07% across our five different machine learning models, including a MAPE of under 0.75% for 18 of the 19 stocks for the best ensemble (boosted regression tree).

An AI-Enabled Stock Prediction Platform Combining News and Social Sensing with Financial Statements

Stock Market Prediction and Performance Analysis Using Machine Learning Algorithms

Predicting short-term stock prices using ensemble methods and online data sources

A stock price prediction method based on deep learning technology

Financial Forecasting of Stock Market Using Sentiment Analysis and Data Analytics

Using dynamic semantic structure of news flow to enhance financial forecasting: a twelve-year study on twitter news channels

Astock: A New Dataset and Automated Stock Trading based on Stock-specific News Analyzing Model

Deep-learning-based stock market prediction incorporating ESG sentiment and technical indicators

Investigating Deep Stock Market Forecasting with Sentiment Analysis

Exploiting Financial News and Social Media Opinions for Stock Market Analysis using MCMC Bayesian Inference

Is Deep-Learning and Natural Language Processing Transcending the Financial Forecasting? Investigation Through Lens of News Analytic Process

Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong

Predicting Financial Market Trends using Time Series Analysis and Natural Language Processing

Incorporating Expert-Based Investment Opinion Signals in Stock Prediction: A Deep Learning Framework.

Harvesting social media sentiment analysis to enhance stock market prediction using deep learning

Stock Market Prediction and Portfolio Management with AI

Harmonizing Macro-Financial Factors and Twitter Sentiment Analysis in Forecasting Stock Market Trends

The Predictive Power of Social Media Sentiment: Evidence from Cryptocurrencies and Stock Markets Using NLP and Stochastic ANNs

Stock Price Predictions with LSTM Neural Networks and Twitter Sentiment

News-Driven Stock Price Forecasting in Indian Markets: A Comparative Study of Advanced Deep Learning Models

Real-Time Online Stock Forecasting Utilizing Integrated Quantitative and Qualitative Analysis