Abstract:Recent efforts have been dedicated to enhancing time series forecasting accuracy by introducing advanced network architectures and self-supervised pretraining strategies. Nevertheless, existing approaches still exhibit two critical drawbacks. Firstly, these methods often rely on a single dataset for training, limiting the model's generalizability due to the restricted scale of the training data. Secondly, the one-step generation schema is widely followed, which necessitates a customized forecasting head and overlooks the temporal dependencies in the output series, and also leads to increased training costs under different horizon length settings. To address these issues, we propose a novel generative pretrained hierarchical transformer architecture for forecasting, named \textbf{GPHT}. There are two aspects of key designs in GPHT. On the one hand, we advocate for constructing a mixed dataset under the channel-independent assumption for pretraining our model, comprising various datasets from diverse data scenarios. This approach significantly expands the scale of training data, allowing our model to uncover commonalities in time series data and facilitating improved transfer to specific datasets. On the other hand, GPHT employs an auto-regressive forecasting approach, effectively modeling temporal dependencies in the output series. Importantly, no customized forecasting head is required, enabling \textit{a single model to forecast at arbitrary horizon settings.} We conduct sufficient experiments on eight datasets with mainstream self-supervised pretraining models and supervised models. The results demonstrated that GPHT surpasses the baseline models across various fine-tuning and zero/few-shot learning settings in the traditional long-term forecasting task. We make our codes publicly available\footnote{<a class="link-external link-https" href="https://github.com/icantnamemyself/GPHT" rel="external noopener nofollow">this https URL</a>}.

Harmonic Recurrent Process for Time Series Forecasting.

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Enhancing Time Series Forecasting: A Hierarchical Transformer with Probabilistic Decomposition Representation

A novel hybrid deep learning time series forecasting model based on long-short-term patterns

Synergetic Learning of Heterogeneous Temporal Sequences for Multi-Horizon Probabilistic Forecasting

Temporal pattern attention for multivariate time series forecasting

Stochastic Recurrent Neural Network for Multistep Time Series Forecasting

A Time Series Forecasting Approach Based on Nonlinear Spiking Neural Systems

A Composite Quantile Fourier Neural Network for Multi-Step Probabilistic Forecasting of Nonstationary Univariate Time Series

N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting

Time series forecasting model for non-stationary series pattern extraction using deep learning and GARCH modeling

TPRNN: A Top-Down Pyramidal Recurrent Neural Network for Time Series Forecasting

Multivariate time series forecasting method based on nonlinear spiking neural P systems and non-subsampled shearlet transform

Deep probabilistic time series forecasting using augmented recurrent input for dynamic systems

Periodicity Decoupling Framework for Long-term Series Forecasting

DPHM-Net:de-redundant multi-period hybrid modeling network for long-term series forecasting

A hybrid framework for multivariate long-sequence time series forecasting

Hidden behavior prediction of complex system based on time-delay belief rule base forecasting model

Generative Pretrained Hierarchical Transformer for Time Series Forecasting

Parallel Extraction of Long-term Trends and Short-term Fluctuation Framework for Multivariate Time Series Forecasting

A machine learning approach for forecasting hierarchical time series