Abstract:Transformer-based models have traditionally been the primary focus of research for addressing time series forecasting challenges. However, the emergence of recently introduced high-performance linear models has cast doubt upon the effectiveness of transformer architecture in time series forecasting tasks. Throughout, most Transformer variants have represented time series using time point-wise tokenization, which does not provide sufficient semantic information for the attention mechanism. PatchTST expands the receptive field through patch-wise tokenization, mitigating the problem of inadequate information. However, when confronted with multivariate time series forecasting tasks, it does not consider the potential impact of delays and correlation between variates on prediction performance. The recently proposed iTransformer addresses the issue of misalignment between variates by employing series-wise tokenization, yet its embedding method is limited to shallow temporal feature representation. In this work, we propose the Temporal Feature Enhanced Transformer (TFEformer), which deeply integrates patch-wise and series-wise tokenization to enhance the temporal representation of multivariate tokens. Furthermore, we introduce a multi-scale patch fusion mechanism capable of capturing and adaptively integrating temporal features across multiple resolutions. We also enhanced the FFN module to serve as a temporal feature extractor and introduced variate-wise attention to capture the correlations between variables. Extensive experiments on eight real-world datasets have demonstrated that TFEformer outperforms all existing models, achieving state-of-the-art performance. Through experiments, we have also shown that TFEformer improves transformer-based models with superior generalization ability, better utilization of extended lookback windows, and effective suppression of distribution shifts.

Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

xPatch: Dual-Stream Time Series Forecasting with Exponential Seasonal-Trend Decomposition

FlexTSF: A Universal Forecasting Model for Time Series with Variable Regularities

Combating Distribution Shift for Accurate Time Series Forecasting via Hypernetworks

Performative Time-Series Forecasting

A Pattern Discovery Approach to Multivariate Time Series Forecasting

Extracting Spatio-Temporal Coupling Feature of Patches for Long-Term Multivariate Time Series Forecasting

ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons

FDF: Flexible Decoupled Framework for Time Series Forecasting with Conditional Denoising and Polynomial Modeling

TFDNet: Time-Frequency Enhanced Decomposed Network for Long-term Time Series Forecasting

CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns

PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the Perspective of Partial Differential Equations

Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting

DRFormer: Multi-Scale Transformer Utilizing Diverse Receptive Fields for Long Time-Series Forecasting

DEPTS: Deep Expansion Learning for Periodic Time Series Forecasting

Phase-Space-Guided Deep Learning for Time Series Forecasting

Adaptive Multi-Scale Decomposition Framework for Time Series Forecasting

Take an Irregular Route: Enhance the Decoder of Time-Series Forecasting Transformer

TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis