Abstract:Transformer-based models have traditionally been the primary focus of research for addressing time series forecasting challenges. However, the emergence of recently introduced high-performance linear models has cast doubt upon the effectiveness of transformer architecture in time series forecasting tasks. Throughout, most Transformer variants have represented time series using time point-wise tokenization, which does not provide sufficient semantic information for the attention mechanism. PatchTST expands the receptive field through patch-wise tokenization, mitigating the problem of inadequate information. However, when confronted with multivariate time series forecasting tasks, it does not consider the potential impact of delays and correlation between variates on prediction performance. The recently proposed iTransformer addresses the issue of misalignment between variates by employing series-wise tokenization, yet its embedding method is limited to shallow temporal feature representation. In this work, we propose the Temporal Feature Enhanced Transformer (TFEformer), which deeply integrates patch-wise and series-wise tokenization to enhance the temporal representation of multivariate tokens. Furthermore, we introduce a multi-scale patch fusion mechanism capable of capturing and adaptively integrating temporal features across multiple resolutions. We also enhanced the FFN module to serve as a temporal feature extractor and introduced variate-wise attention to capture the correlations between variables. Extensive experiments on eight real-world datasets have demonstrated that TFEformer outperforms all existing models, achieving state-of-the-art performance. Through experiments, we have also shown that TFEformer improves transformer-based models with superior generalization ability, better utilization of extended lookback windows, and effective suppression of distribution shifts.

LagCNN: A Fast Yet Effective Model for Multivariate Long-term Time Series Forecasting

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

TimeCNN: Refining Cross-Variable Interaction on Time Point for Time Series Forecasting

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

TCCT: Tightly-coupled convolutional transformer on time series forecasting

CNformer: a convolutional transformer with decomposition for long-term multivariate time series forecasting

Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting

LHCnn - A Novel Efficient Multivariate Time Series Prediction Framework Utilizing Convolutional Neural Networks.

Financial Time Series Forecasting using CNN and Transformer

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

GCformer: an Efficient Solution for Accurate and Scalable Long-Term Multivariate Time Series Forecasting

GCformer: An Efficient Framework for Accurate and Scalable Long-Term Multivariate Time Series Forecasting

AGCNT: Adaptive Graph Convolutional Network for Transformer-based Long Sequence Time-Series Forecasting

Multivariate Time Series Modeling and Forecasting with Parallelized Convolution and Decomposed Sparse-Transformer

CLformer: Locally grouped auto-correlation and convolutional transformer for long-term multivariate time series forecasting

MP3Net:Multi-scale Patch Parallel Prediction Networks for Multivariate Time Series Forecasting

Fast and highly coupled model for time series forecasting

CrossWaveNet: A dual-channel network with deep cross-decomposition for Long-term Time Series Forecasting

PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting

CVTN: Cross Variable and Temporal Integration for Time Series Forecasting