Abstract:Deep probabilistic time series forecasting has gained attention for its ability to provide nonlinear approximation and valuable uncertainty quantification for decision-making. However, existing models often oversimplify the problem by assuming a time-independent error process and overlooking serial correlation. To overcome this limitation, we propose an innovative training method that incorporates error autocorrelation to enhance probabilistic forecasting accuracy. Our method constructs a mini-batch as a collection of $D$ consecutive time series segments for model training. It explicitly learns a time-varying covariance matrix over each mini-batch, encoding error correlation among adjacent time steps. The learned covariance matrix can be used to improve prediction accuracy and enhance uncertainty quantification. We evaluate our method on two different neural forecasting models and multiple public datasets. Experimental results confirm the effectiveness of the proposed approach in improving the performance of both models across a range of datasets, resulting in notable improvements in predictive accuracy.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in deep probabilistic time - series forecasting, existing models usually assume that the error process is independent of time and ignore the serial correlation, which leads to insufficient prediction accuracy. The author proposes a new training method to improve the accuracy of probabilistic prediction by explicitly learning the time - varying covariance matrix in each mini - batch to capture the error autocorrelation.
Specifically, the main contributions of the paper include:
1. **Proposing a new method**: Enhancing the training and prediction of univariate probabilistic time - series models by learning a time - varying covariance matrix. This covariance matrix can capture the correlated errors within the mini - batch.
2. **Parameterizing the dynamic correlation matrix**: Using the weighted sum of several basis kernel matrices to parameterize the dynamic correlation matrix, ensuring that the correlation matrix is a positive - definite symmetric matrix with diagonal elements of 1. This method allows for the joint learning of dynamic weights and the base model.
3. **Experimental verification**: Evaluating the effectiveness of the proposed method on two different neural prediction models (DeepAR and Transformer) and multiple public datasets. The experimental results show that this method significantly improves the prediction accuracy on a variety of datasets.
### Paper Background
Time - series forecasting is an important task in the field of deep learning and is widely used in various scenarios. Time - series forecasting can be divided into deterministic forecasting and probabilistic forecasting. Deterministic forecasting provides point estimates of future time - series values, while probabilistic forecasting further provides a distribution, quantifying the uncertainty of the prediction. Probabilistic forecasting has become increasingly popular because it provides additional uncertainty information, which helps users make more informed decisions.
### Existing Problems
Existing probabilistic time - series prediction models usually assume that the error process is independent of time and ignore the serial correlation. This simplified assumption may lead to insufficient prediction accuracy. In particular, in time - series analysis, errors may exhibit autocorrelation (that is, there is a correlation between errors at different time points), which will affect the performance of the model.
### Solutions
To overcome this limitation, the author proposes a new training method to train the model by constructing mini - batches containing multiple consecutive time - series segments. In each mini - batch, a time - varying covariance matrix is explicitly learned to encode the error correlation between adjacent time steps. The specific steps are as follows:
1. **Constructing mini - batches**: Each mini - batch contains multiple consecutive time - series segments, and the length of each segment is \( P + 1 \) (where \( P \) is the conditional range and 1 is the prediction range).
2. **Learning the covariance matrix**: In each mini - batch, a time - varying covariance matrix is explicitly learned to capture the error autocorrelation.
3. **Optimizing the objective function**: Optimizing the model parameters by maximizing the log - likelihood function, taking into account the autocorrelation of the error.
### Experimental Results
The author conducted experiments on multiple public datasets, and the results show that the proposed method significantly improves the prediction accuracy on multiple datasets. Specifically, compared with the traditional model trained based on Gaussian - likelihood loss, the new method has achieved significant relative improvements on multiple datasets.
### Conclusions
This paper significantly improves the accuracy of probabilistic time - series prediction by introducing a new training method to explicitly model the error autocorrelation. This method not only enhances the training flexibility of the model but also provides high - quality uncertainty quantification.