Abstract:Long-term forecasting is of paramount importance in numerous scenarios, including predicting future energy, water, and food consumption. For instance, extreme weather events and natural disasters can profoundly impact infrastructure operations and pose severe safety concerns. Traditional CNN-based models often struggle to capture long-distance dependencies effectively. In contrast, Transformers-based models have shown significant promise in long-term forecasting. This paper investigates the long-term forecasting problem and identifies a common limitation in existing Transformer-based models: they tend to reduce computational complexity at the expense of time information aggregation capability. Moreover, the order of time series plays a crucial role in accurate predictions, but current Transformer-based models lack sensitivity to time series order, rendering them unreasonable. To address these issues, we propose a novel Deformable-Local (DL) aggregation mechanism. This mechanism enhances the model's ability to aggregate time information and allows the model to adaptively adjust the size of the time aggregation window. Consequently, the model can discern more complex time patterns, leading to more accurate predictions. Additionally, our model incorporates a Frequency Selection module to reinforce effective features and reduce noise. Furthermore, we introduce Position Weights to mitigate the order-insensitivity problem present in existing methods. In extensive evaluations of long-term forecasting tasks, we conducted benchmark tests on six datasets covering various practical applications, including energy, traffic, economics, weather, and disease. Our method achieved state-of-the-art (SOTA) results, demonstrating significant improvements. For instance, on the ETT dataset, our model achieved an average MSE improvement of approximately 19% and an average MAE improvement of around 27%. Remarkably, for predicted lengths of 96 and 192, we achieved outstanding MSE and MAE improvements of 32.1% and 30.9%, respectively.

Cascaded Teaching Transformers with Data Reweighting for Long Sequence Time-series Forecasting

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

Enhancing Time Series Forecasting: A Hierarchical Transformer with Probabilistic Decomposition Representation

Multi-resolution Time-Series Transformer for Long-term Forecasting

Bidformer: A Transformer-Based Model Via Bidirectional Sparse Self-Attention Mechanism for Long Sequence Time-Series Forecasting

Replacing self-attentions with convolutional layers in multivariate long sequence time-series forecasting

RSMformer: an efficient multiscale transformer-based framework for long sequence time-series forecasting

Enhancing Transformer-based models for Long Sequence Time Series Forecasting via Structured Matrix

Leveraging 2D Information for Long-term Time Series Forecasting with Vanilla Transformers

Sparse transformer with local and seasonal adaptation for multivariate time series forecasting

Itransformer: Inverted Transformers Are Effective for Time Series Forecasting

A Time Series is Worth 64 Words: Long-term Forecasting with Transformers

DRFormer: Multi-Scale Transformer Utilizing Diverse Receptive Fields for Long Time-Series Forecasting

sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting

Resformer: Combine quadratic linear transformation with efficient sparse Transformer for long-term series forecasting

PWDformer: Deformable Transformer for Long-Term Series Forecasting

InParformer: Evolutionary Decomposition Transformers with Interactive Parallel Attention for Long-Term Time Series Forecasting

Sparse-VQ Transformer: An FFN-Free Framework with Vector Quantization for Enhanced Time Series Forecasting

GRAformer: A Gated Residual Attention Transformer for Multivariate Time Series Forecasting