Gated three-tower transformer for text-driven stock market prediction
Jia Chen,Tao Chen,Mengqi Shen,Yunhai Shi,Dongjing Wang,Xin Zhang
DOI: https://doi.org/10.1007/s11042-022-11908-1
IF: 2.577
2022-04-05
Multimedia Tools and Applications
Abstract:Effective stock market prediction can significantly assist individual and institutional investors to make better trading decisions and help government stabilize the market. Therefore, a variety of methods have been proposed to tackle the issue of stock market prediction recently. However, it is still quite challenging to effectively extract the correlations and temporal information from multivariate time series of market data and integrate various kinds of features as well as auxiliary information, which is important for improving the performance of stock market prediction. This paper proposes an entirely Transformer based model, namely Gated Three-Tower Transformer (GT3), to incorporate numerical market information and social text information for accurate stock market prediction. Firstly, we devise a Channel-Wise Tower Encoder (CWTE) to capture the channel-wise features from transposed numerical data embeddings. Secondly, we design a Shifted Window Tower Encoder (SWTE) with Multi-Temporal Aggregation to extract and aggregate the multi-scale temporal features from the original numerical data embeddings. Then we adopt the encoder of vanilla Transformer as a Text Tower Encoder (TTE) to obtain the high-level textual features. Furthermore, we design a Cross-Tower Attention mechanism to assist the model to learn the trend-relevant significance of each daily text representation by leveraging the temporal features from SWTE. Finally, we unify CWTE, SWTE, and TTE as the GT3 model through a self-adaptive gate layer to perform end-to-end text-driven stock market prediction by fusing three types of features effectively and efficiently. Extensive experimental results on a real-world dataset show that the proposed model outperforms state-of-the-art baselines.
computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering