Abstract:Machine learning often lacks transparent performance indicators, especially in generating point predictions. This paper addresses this limitation through conformal prediction, a non-parametric forecasting technique seamlessly integrating with regression algorithms to produce prediction intervals at specified confidence levels. A crucial element in conformal prediction is the non-conformity score, traditionally based on absolute residual errors. In this work, we propose a novel approach, introducing data-dependent weights for computing non-conformity scores. This enhancement, considering the distances of training instances from the test sample, aims to improve overall algorithm performance. Empirical investigations across various real-world regression data sets, including scientific data, evaluate the efficiency and validity of prediction intervals from different uncertainty quantification methods. Results show that prediction intervals computed with data-dependent weights adapt to estimator uncertainty, offering more precise predictions in certain scenarios and appropriately conservative predictions in high uncertainty situations. Additionally, we compare predictive regions generated by conformal prediction with those from Gaussian Process Regression (GPR) for scientific data in structural engineering. To augment conformal prediction, we explore Conformalized Quantile Regression (CQR), a recent innovation combining conformal prediction with classical quantile regression, claiming full adaptability to heteroscedasticity. Our findings indicate that conformal prediction methods using data-dependent non-conformity scores achieve a 1% higher effective coverage level and a 15% reduction in prediction interval widths compared to other methods. The comparative analysis against GPR and CQR underscores the practical value of our approach in providing accurate prediction intervals in scientific and engineering domains.

Asymptotic analysis of locally weighted jackknife prediction

Probabilistic prediction with locally weighted jackknife predictive system

Predictive inference with the jackknife+

Conformal prediction with local weights: randomization enables local guarantees

Adaptive Conformal Regression with Jackknife+ Rescaled Scores

Kernel-based optimally weighted conformal prediction intervals

Group-Weighted Conformal Prediction

On Nonsmooth Estimating Functions via Jackknife Empirical Likelihood

Conformal Prediction using Conditional Histograms

Jackknife Model Averaging for Mixed-Data Kernel-Weighted Spline Quantile Regressions

Conformal prediction for robust deep nonparametric regression

JAWS: Auditing Predictive Uncertainty Under Covariate Shift

Central Limit Theorems of a Recursive Stochastic Algorithm with Applications to Adaptive Designs

Building Conformal Prediction Intervals with Approximate Message Passing

Asymptotics of nonparametric L-1 regression models with dependent data

Jackknife Empirical Likelihood for the Error Variance in Linear Models

Penalized Jackknife Empirical Likelihood in High Dimensions

General Jackknife Empirical Likelihood and Its Applications.

Spatial Conformal Inference through Localized Quantile Regression

Regression Conformal Prediction under Bias

Adaptive Conformal Prediction Intervals Using Data-Dependent Weights With Application to Seismic Response Prediction