Abstract:We present a conformal prediction method for time series using the Transformer architecture to capture long-memory and long-range dependencies. Specifically, we use the Transformer decoder as a conditional quantile estimator to predict the quantiles of prediction residuals, which are used to estimate the prediction interval. We hypothesize that the Transformer decoder benefits the estimation of the prediction interval by learning temporal dependencies across past prediction residuals. Our comprehensive experiments using simulated and real data empirically demonstrate the superiority of the proposed method compared to the existing state-of-the-art conformal prediction methods.

What problem does this paper attempt to address?

The paper is primarily dedicated to addressing the issue of uncertainty quantification in time series forecasting, specifically how to construct effective prediction intervals for the predictions of machine learning models. Specifically, the authors propose a time series Conformal Prediction method based on the Transformer architecture to handle the common long-term dependencies and strong temporal correlations in time series data. ### Research Background and Challenges - **Uncertainty Quantification**: With the widespread application of black-box machine learning models in various fields, uncertainty quantification has become crucial. - **Conformal Prediction**: As a popular uncertainty quantification technique, it addresses this challenge by providing effective predictive inference for black-box models. - **Time Series Forecasting**: Aims to predict future values based on a series of observations arranged in chronological order. - **Challenges**: The challenges of applying conformal prediction to time series data include violations of the exchangeability assumption, significant random fluctuations, and strong temporal correlations. ### Main Contributions - **Method Innovation**: Utilizes the Transformer decoder as a conditional quantile estimator to predict the quantiles of prediction residuals, thereby estimating the prediction intervals. - **Advantages**: The Transformer decoder can effectively capture long-term dependencies and can leverage additional features for conditional quantile estimation, allowing the model to learn the potential dependencies between these features and the residuals. - **Experimental Validation**: Extensive experiments on simulated data and real datasets demonstrate that the proposed SPCI-T method outperforms existing state-of-the-art conformal prediction methods. ### Experimental Results - **Simulated Data**: Under both non-stationary and heteroscedastic error conditions, the SPCI-T method achieves narrower prediction interval widths while maintaining good coverage. - **Real Data**: Tested on three real-world datasets—solar radiation, wind speed records, and electricity usage—the SPCI-T method performs excellently, with performance further enhanced when additional features are included. In summary, this study proposes a novel and effective solution to the problem of conformal prediction in time series forecasting, demonstrating its advantages in handling complex time series data.

Transformer Conformal Prediction for Time Series

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution

Enhancing Time Series Forecasting: A Hierarchical Transformer with Probabilistic Decomposition Representation

Dateformer: Transformer Extends Look-back Horizon to Predict Longer-term Time Series

Conformal Asymmetric Multi-Quantile Generative Transformer for Day-Ahead Wind Power Interval Prediction

Dateformer: Time-modeling Transformer for Longer-term Series Forecasting

Itransformer: Inverted Transformers Are Effective for Time Series Forecasting

Segmented Frequency‐Domain Correlation Prediction Model for Long‐Term Time Series Forecasting Using Transformer

Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting

Probabilistic Decomposition Transformer for Time Series Forecasting

Expanding the Prediction Capacity in Long Sequence Time-Series Forecasting

CLformer: Locally grouped auto-correlation and convolutional transformer for long-term multivariate time series forecasting

Are Transformers Effective for Time Series Forecasting?

Physically-guided Temporal Diffusion Transformer for Long-Term Time Series Forecasting

sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting

NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting

A Differential Attention Fusion Model Based on Transformer for Time Series Forecasting

Inter-Series Transformer: Attending to Products in Time Series Forecasting