Abstract:We study the split Conformal Prediction method when applied to Markovian data. We quantify the gap in terms of coverage induced by the correlations in the data (compared to exchangeable data). This gap strongly depends on the mixing properties of the underlying Markov chain, and we prove that it typically scales as $\sqrt{t_\mathrm{mix}\ln(n)/n}$ (where $t_\mathrm{mix}$ is the mixing time of the chain). We also derive upper bounds on the impact of the correlations on the size of the prediction set. Finally we present $K$-split CP, a method that consists in thinning the calibration dataset and that adapts to the mixing properties of the chain. Its coverage gap is reduced to $t_\mathrm{mix}/(n\ln(n))$ without really affecting the size of the prediction set. We finally test our algorithms on synthetic and real-world datasets.
What problem does this paper attempt to address?
This paper attempts to address the problem of how to quantify the coverage probability gap caused by data correlation when applying the split Conformal Prediction (CP) method under Markovian data, and proposes improved methods to adapt to these correlations.
Specifically, the paper focuses on the following two core issues:
1. **The impact of data correlation on coverage probability and prediction set size**: Traditional split Conformal Prediction methods assume that calibration data are independent and identically distributed (i.i.d.) or at least exchangeable. However, in practical applications, there may be high correlations between data samples, especially in learning tasks involving time series and dynamic systems. This correlation affects the coverage probability of split Conformal Prediction and the size of the prediction set. The paper aims to quantify these impacts and analyze the relationship between these impacts and the mixing properties of the underlying Markov chain.
2. **How to adapt to data correlation to reduce the coverage probability gap**: To reduce the impact of data correlation on split Conformal Prediction methods, the paper proposes the K-split Conformal Prediction (K-split CP) method. This method mitigates the impact of correlation by sparsifying the calibration dataset, thereby reducing the coverage probability gap without significantly increasing the size of the prediction set.
### Main Contributions
1. **Theoretical Analysis**: The paper provides a theoretical analysis of the marginal coverage probability and prediction set size of split Conformal Prediction methods under Markovian data. The study shows that the additional coverage probability gap caused by data correlation depends on the mixing properties of the underlying Markov chain, and under general ergodicity assumptions, this gap usually does not exceed \(\sqrt{\frac{t_{\text{mix}} \ln(n)}{n}}\), where \(t_{\text{mix}}\) is the mixing time of the Markov chain, and \(n\) is the size of the calibration dataset. Additionally, the increase in prediction interval size due to correlation usually does not exceed \(\sqrt{\frac{t_{\text{mix}}}{n}}\).
2. **K-split Conformal Prediction Method**: The paper proposes the K-split Conformal Prediction method, which reduces the impact of correlation by sparsifying the calibration dataset. The optimized K value can be estimated online without significantly affecting the coverage probability. The K-split Conformal Prediction method reduces the coverage probability gap from \(\sqrt{\frac{t_{\text{mix}} \ln(n)}{n}}\) to \(\frac{t_{\text{mix}}}{n \ln(n)}\), with minimal impact on the size of the prediction interval.
3. **Numerical Experiment Validation**: The paper validates the theoretical results through numerical experiments, including applications on synthetic data and real-world data (such as EUR/SEK exchange rate prediction).
### Related Work
The paper also reviews related research, including work on extending the Conformal Prediction framework to handle non-exchangeable data and methods for adjusting empirical quantile levels to account for coverage deficiencies that may arise from correlation. These studies provide theoretical foundations and methodological support for the paper.
### Conclusion
Through theoretical analysis and experimental validation, the paper demonstrates how to quantify and reduce the impact of data correlation on coverage probability and prediction set size when applying split Conformal Prediction methods under Markovian data. The proposed K-split Conformal Prediction method provides an effective solution for handling correlated data.