Abstract:Transformer-based models have traditionally been the primary focus of research for addressing time series forecasting challenges. However, the emergence of recently introduced high-performance linear models has cast doubt upon the effectiveness of transformer architecture in time series forecasting tasks. Throughout, most Transformer variants have represented time series using time point-wise tokenization, which does not provide sufficient semantic information for the attention mechanism. PatchTST expands the receptive field through patch-wise tokenization, mitigating the problem of inadequate information. However, when confronted with multivariate time series forecasting tasks, it does not consider the potential impact of delays and correlation between variates on prediction performance. The recently proposed iTransformer addresses the issue of misalignment between variates by employing series-wise tokenization, yet its embedding method is limited to shallow temporal feature representation. In this work, we propose the Temporal Feature Enhanced Transformer (TFEformer), which deeply integrates patch-wise and series-wise tokenization to enhance the temporal representation of multivariate tokens. Furthermore, we introduce a multi-scale patch fusion mechanism capable of capturing and adaptively integrating temporal features across multiple resolutions. We also enhanced the FFN module to serve as a temporal feature extractor and introduced variate-wise attention to capture the correlations between variables. Extensive experiments on eight real-world datasets have demonstrated that TFEformer outperforms all existing models, achieving state-of-the-art performance. Through experiments, we have also shown that TFEformer improves transformer-based models with superior generalization ability, better utilization of extended lookback windows, and effective suppression of distribution shifts.

RTFN: A robust temporal feature network for time series classification

RTFN: Robust Temporal Feature Network

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Adaptive Multi-Scale TF-net for High-Resolution Time-Frequency Representations

QTFN: A General End-to-End Time-Frequency Network to Reveal the Time-Varying Signatures of the Time Series

RNTS: Robust Neural Temporal Search for Time Series Classification

CTFNet: Long-Sequence Time-Series Forecasting Based on Convolution and Time–Frequency Analysis

LSTM-MFCN: A time series classifier based on multi-scale spatial–temporal features

Time Series Classification with Random Temporal Features

Adaptive feature fusion for time series classification

Time Series Classification Via Enhanced Temporal Representation Learning

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

Rel-CNN: Learning Relationship Features in Time Series for Classification

Combining contextual neural networks for time series classification

Temporal Representation Learning for Time Series Classification.

Attention-based LSTM-CNNs for Time-series Classification.

Heterogeneous Feature Based Time Series Classification With Attention Mechanism

Ensemble Based Fully Convolutional Transformer Network for Time Series Classification

TimeNet: Pre-trained deep recurrent neural network for time series classification

Multivariate Time Series Classification with Crucial Timestamps Guidance

TFDNet: Time-Frequency Enhanced Decomposed Network for Long-term Time Series Forecasting