Abstract:Transformer-based models have traditionally been the primary focus of research for addressing time series forecasting challenges. However, the emergence of recently introduced high-performance linear models has cast doubt upon the effectiveness of transformer architecture in time series forecasting tasks. Throughout, most Transformer variants have represented time series using time point-wise tokenization, which does not provide sufficient semantic information for the attention mechanism. PatchTST expands the receptive field through patch-wise tokenization, mitigating the problem of inadequate information. However, when confronted with multivariate time series forecasting tasks, it does not consider the potential impact of delays and correlation between variates on prediction performance. The recently proposed iTransformer addresses the issue of misalignment between variates by employing series-wise tokenization, yet its embedding method is limited to shallow temporal feature representation. In this work, we propose the Temporal Feature Enhanced Transformer (TFEformer), which deeply integrates patch-wise and series-wise tokenization to enhance the temporal representation of multivariate tokens. Furthermore, we introduce a multi-scale patch fusion mechanism capable of capturing and adaptively integrating temporal features across multiple resolutions. We also enhanced the FFN module to serve as a temporal feature extractor and introduced variate-wise attention to capture the correlations between variables. Extensive experiments on eight real-world datasets have demonstrated that TFEformer outperforms all existing models, achieving state-of-the-art performance. Through experiments, we have also shown that TFEformer improves transformer-based models with superior generalization ability, better utilization of extended lookback windows, and effective suppression of distribution shifts.

TCF-Trans: Temporal Context Fusion Transformer for Anomaly Detection in Time Series

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

TFD-Net: Transformer Deviation Network for Weakly Supervised Anomaly Detection

Multivariate Time Series Anomaly Detection with Fourier Time Series Transformer

Concept Drift Adaptation for Time Series Anomaly Detection via Transformer

Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy

Decompose Auto-Transformer Time Series Anomaly Detection for Network Management

DDMT: Denoising Diffusion Mask Transformer Models for Multivariate Time Series Anomaly Detection

DTAAD: Dual Tcn-attention networks for anomaly detection in multivariate time series data

Local-Adaptive Transformer for Multivariate Time Series Anomaly Detection and Diagnosis.

Time Series Anomaly Detection with a Transformer Residual Autoencoder-Decoder.

Sub-Adjacent Transformer: Improving Time Series Anomaly Detection with Reconstruction Error from Sub-Adjacent Neighborhoods

MPFormer: Multi-Patch Transformer for Multivariate Time Series Anomaly Detection with Contrastive Learning

Temporal-Frequency Masked Autoencoders for Time Series Anomaly Detection

BTAD: A binary transformer deep neural network model for anomaly detection in multivariate time series data

SiET: Spatial information enhanced transformer for multivariate time series anomaly detection

UTRAD: Anomaly detection and localization with U-Transformer

A Time Series Anomaly Detection Method Based on Series-Parallel Transformers with Spatial and Temporal Association Discrepancies

Weakly Supervised Video Anomaly Detection Via Self-Guided Temporal Discriminative Transformer.

InfoFlow: A Transformer-based Time Series Anomaly Detection Model with Information Bottleneck and Normalizing Flows