Accurate and Reliable Forecasting using Stochastic Differential Equations

Peng Cui,Zhijie Deng,Wenbo Hu,Jun Zhu
DOI: https://doi.org/10.48550/arXiv.2103.15041
2021-03-28
Abstract:It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments. Although a lot of efforts have been made, such as heteroscedastic neural networks (HNNs), little work has demonstrated satisfactory practicability due to the different levels of compromise on learning efficiency, quality of uncertainty estimates, and predictive performance. Moreover, existing HNNs typically fail to construct an explicit interaction between the prediction and its associated uncertainty. This paper aims to remedy these issues by developing SDE-HNN, a new heteroscedastic neural network equipped with stochastic differential equations (SDE) to characterize the interaction between the predictive mean and variance of HNNs for accurate and reliable regression. Theoretically, we show the existence and uniqueness of the solution to the devised neural SDE. Moreover, based on the bias-variance trade-off for the optimization in SDE-HNN, we design an enhanced numerical SDE solver to improve the learning stability. Finally, to more systematically evaluate the predictive uncertainty, we present two new diagnostic uncertainty metrics. Experiments on the challenging datasets show that our method significantly outperforms the state-of-the-art baselines in terms of both predictive performance and uncertainty quantification, delivering well-calibrated and sharp prediction intervals.
Machine Learning
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is: how to accurately and reliably characterize the uncertainty that is ubiquitous in the real - world environment in deep - learning models. Specifically, the author points out the deficiencies of existing heteroscedastic neural networks (HNNs) in terms of learning efficiency, the quality of uncertainty estimation, and prediction performance, and proposes a new method - SDE - HNN (stochastic differential equation - based heteroscedastic neural network) to improve these aspects. ### Core Problems of the Paper 1. **Uncertainty Quantification**: - Although existing deep - learning models perform well on many tasks, when dealing with real - world uncertainty, they often fail to provide reliable prediction intervals (PIs). This makes it difficult to trust the models in practical applications (such as autonomous driving, medical diagnosis, etc.). 2. **Interaction between Prediction and Uncertainty**: - Existing HNNs usually fail to establish an explicit interaction between the predicted value and its associated uncertainty. This lack of interaction leads to a weak coupling between prediction and uncertainty, affecting the reliability and accuracy of the model. 3. **Bias - Variance Trade - off in the Optimization Process**: - In the process of jointly optimizing the predicted mean and variance, there is a problem of bias - variance trade - off, which may lead to training instability, especially when approaching convergence. ### Solutions To solve the above problems, the author proposes SDE - HNN, which establishes an explicit interaction between the predicted mean and variance by introducing stochastic differential equations (SDE). The specific contributions are as follows: 1. **Theoretical Guarantee**: - Prove the existence and uniqueness of the designed neural SDE solution (Equation (9)). 2. **Improvement of Numerical Solver**: - Based on the bias - variance trade - off, design an improved Euler - Maruyama method (Equation (10)) to improve learning stability. 3. **New Diagnostic Indicators**: - Propose two new diagnostic uncertainty measurement indicators: confidence - weighted calibration error (CWCE) and its variant R - CWCE (Equation (12)), for more systematic evaluation of prediction uncertainty. ### Experimental Results Experiments show that SDE - HNN significantly outperforms existing baseline methods on multiple challenging datasets, not only improving prediction performance but also providing high - quality prediction intervals. The following are some of the experimental result comparisons: | Dataset | Indicator | MCD | DGP | BNN | Deep - ens | HNN | Proposed | | --- | --- | --- | --- | --- | --- | --- | --- | | Metro - traffic | RMSE | 697.021 | 651.341 | 786.694 | 533.426 | 559.354 | 483.639±2.657 | | | CWCE | 52.152 | 10.552 | 21.486 | 9.078 | 9.305 | 2.894±0.085 | | Pickups | RMSE | 625.812 | 523.041 | 720.013 | 428.032 | 421.752 | 340.331±5.072 | | | CWCE | 34.441 | 22.799 | 42.570 | 4.878 | 6.043 | 2.925±0.758 | In summary, this paper aims to achieve more accurate and reliable regression predictions by introducing SDE - HNN, and has made significant progress especially in uncertainty quantification.