Abstract:Time series forecasting presents a significant challenge, particularly when its accuracy relies on external data sources rather than solely on historical values. This issue is prevalent in the financial sector, where the future behavior of time series is often intricately linked to information derived from various textual reports and a multitude of economic indicators. In practice, the key challenge lies in constructing a reliable time series forecasting model capable of harnessing data from diverse sources and extracting valuable insights to predict the target time series accurately. In this work, we tackle this challenging problem and introduce a novel multimodal transformer-based model named the \textit{Modality-aware Transformer}. Our model excels in exploring the power of both categorical text and numerical timeseries to forecast the target time series effectively while providing insights through its neural attention mechanism. To achieve this, we develop feature-level attention layers that encourage the model to focus on the most relevant features within each data modality. By incorporating the proposed feature-level attention, we develop a novel Intra-modal multi-head attention (MHA), Inter-modal MHA and Target-modal MHA in a way that both feature and temporal attentions are incorporated in MHAs. This enables the MHAs to generate temporal attentions with consideration of modality and feature importance which leads to more informative embeddings. The proposed modality-aware structure enables the model to effectively exploit information within each modality as well as foster cross-modal understanding. Our extensive experiments on financial datasets demonstrate that Modality-aware Transformer outperforms existing methods, offering a novel and practical solution to the complex challenges of multi-modal financial time series forecasting.

Higher Order Transformers: Enhancing Stock Movement Prediction On Multimodal Time-Series Data

Incorporating Transformers and Attention Networks for Stock Movement Prediction

Spatiotemporal Transformer for Stock Movement Prediction

Deep Convolutional Transformer Network for Stock Movement Prediction

Higher Order Transformers: Efficient Attention Mechanism for Tensor Structured Data

Support for Stock Trend Prediction Using Transformers and Sentiment Analysis

Transformers with Attentive Federated Aggregation for Time Series Stock Forecasting

Kernel-based Hybrid Interpretable Transformer for High-frequency Stock Movement Prediction

HWSformer: History Window Serialization Based Transformer for Semantic Enrichment Driven Stock Market Prediction

Transformer-based Reinforcement Learning Model for Optimized Quantitative Trading

Stock market index prediction using transformer neural network models and frequency decomposition

Stock Price Prediction Based on Non-Stationary Transformers Model

An End-to-End Structure with Novel Position Mechanism and Improved EMD for Stock Forecasting

Financial Time Series Forecasting using CNN and Transformer

Modality-aware Transformer for Financial Time series Forecasting

HATR-I: Hierarchical Adaptive Temporal Relational Interaction for Stock Trend Prediction

Ensemble of temporal Transformers for financial time series

Adaptive Long-Short Pattern Transformer for Stock Investment Selection

Gated three-tower transformer for text-driven stock market prediction

From attention to profit: quantitative trading strategy based on transformer

T2V_TF: An adaptive timing encoding mechanism based Transformer with multi-source heterogeneous information fusion for portfolio management: A case of the Chinese A50 stocks