Abstract:Long sequence time-series forecasting (LSTF) problems, such as weather forecasting, stock market forecasting, and power resource management, are widespread in the real world. The LSTF problem requires a model with high prediction accuracy. Recent studies have shown that the transformer model architecture is the most promising model structure for LSTF problems compared with other model architectures. The transformer model has the property of permutation equivalence, which leads to the importance of sequence position encoding, an essential process in model training. Currently, the continuous dynamics models constructed for position encoding using the neural differential equations (neural ODEs) method can model sequence position information well. However, we have found that there are some limitations when neural ODEs are applied to the LSTF problem, including the time cost problem, the baseline drift problem, and the information loss problem; thus, neural ODEs cannot be directly applied to the LSTF problem. To address this problem, we design a binary position encoding-based regularization model for long sequence time-series prediction, named Seformer, which has the following structure: 1) The binary position encoding mechanism, including intrablock and interblock position encoding. For intrablock position encoding, we design a simple ODE method by discretizing the continuum dynamics model, which reduces the time cost required to compute neural ODEs while maintaining their dynamics properties to the maximum extent. In interblock position encoding, a chunked recursive form is adopted to alleviate the baseline drift problem caused by eigenvalue explosion. 2) Information transfer regularization mechanism: By regularizing the model intermediate hidden variables as well as the encoder-decoder connection variables, we can reduce information loss during the model training process while ensuring the smoothness of the position information. Extensive experimental results obtained on six large-scale datasets show a consistent improvement in our approach over the baselines.

CL-Informer: Long time series prediction model based on continuous wavelet transform

MSRN-Informer: Time Series Prediction Model Based on Multi-Scale Residual Network

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Time Series Analysis Based on Informer Algorithms: A Survey

Short-term wind power forecasting model based on temporal convolutional network and Informer

Adaptive Graph Convolutional Network Framework for Multidimensional Time Series Prediction

Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

Short-term Prediction of Offshore Wind Power Based on CNN and LSTM

CLformer: Locally grouped auto-correlation and convolutional transformer for long-term multivariate time series forecasting

Power-Load Forecasting Model Based on Informer and Its Application

PMformer: A novel informer-based model for accurate long-term time series prediction

Enhanced LFTSformer: A Novel Long-Term Financial Time Series Prediction Model Using Advanced Feature Engineering and the DS Encoder Informer Architecture

A Hybrid Forecasting Model Based on CNN and Informer for Short-Term Wind Power

Stock and market index prediction using Informer network

MFTM-Informer: A Multi-Step Prediction Model Based on Multivariate Fuzzy Trend Matching and Informer

Expanding the Prediction Capacity in Long Sequence Time-Series Forecasting

A hybrid framework for multivariate long-sequence time series forecasting

Seformer: a Long Sequence Time-Series Forecasting Model Based on Binary Position Encoding and Information Transfer Regularization

GCNInformer: A combined deep learning model based on GCN and Informer for wind power forecasting

Model for Non-Periodic Time Series Prediction

Financial Time Series Image Algorithm Based on Wavelet Analysis and Data Fusion