Abstract:Transformer-based models have traditionally been the primary focus of research for addressing time series forecasting challenges. However, the emergence of recently introduced high-performance linear models has cast doubt upon the effectiveness of transformer architecture in time series forecasting tasks. Throughout, most Transformer variants have represented time series using time point-wise tokenization, which does not provide sufficient semantic information for the attention mechanism. PatchTST expands the receptive field through patch-wise tokenization, mitigating the problem of inadequate information. However, when confronted with multivariate time series forecasting tasks, it does not consider the potential impact of delays and correlation between variates on prediction performance. The recently proposed iTransformer addresses the issue of misalignment between variates by employing series-wise tokenization, yet its embedding method is limited to shallow temporal feature representation. In this work, we propose the Temporal Feature Enhanced Transformer (TFEformer), which deeply integrates patch-wise and series-wise tokenization to enhance the temporal representation of multivariate tokens. Furthermore, we introduce a multi-scale patch fusion mechanism capable of capturing and adaptively integrating temporal features across multiple resolutions. We also enhanced the FFN module to serve as a temporal feature extractor and introduced variate-wise attention to capture the correlations between variables. Extensive experiments on eight real-world datasets have demonstrated that TFEformer outperforms all existing models, achieving state-of-the-art performance. Through experiments, we have also shown that TFEformer improves transformer-based models with superior generalization ability, better utilization of extended lookback windows, and effective suppression of distribution shifts.

TEST-Net: transformer-enhanced Spatio-temporal network for infectious disease prediction

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Predicting the Epidemics Trend of COVID-19 Using Epidemiological-Based Generative Adversarial Networks.

Integrating Transformer and GCN for COVID-19 Forecasting

STG-Net: A COVID-19 prediction network based on multivariate spatio-temporal information

Deep evolutionary fusion neural network: a new prediction standard for infectious disease incidence rates

Inter- and Intra-Series Embeddings Fusion Network for Epidemiological Forecasting

A Novel Multi-scale Spatiotemporal Graph Neural Network for Epidemic Prediction.

A Neural Network Framework for Fine-Grained Tropical Cyclone Intensity Prediction

NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting

Prediction of Spread Trend of Epidemic Based on Spatial-Temporal Sequence

High-resolution short-term prediction of the COVID-19 epidemic based on spatial-temporal model modified by historical meteorological data

Spatial-Temporal Synchronous Graph Transformer network (STSGT) for COVID-19 forecasting

Spatio-Temporal Transformer Network for Weather Forecasting

Long-term prediction for temporal propagation of seasonal influenza using Transformer-based model

Interpretable spatial identity neural network-based epidemic prediction

Long-term trend prediction of pandemic combining the compartmental and deep learning models

Research on the regional prediction model of urban raster infectious diseases based on deep learning

Predicting Infectious Disease Using Deep Learning and Big Data

A time series driven model for early sepsis prediction based on transformer module