Abstract:Machine learning often lacks transparent performance indicators, especially in generating point predictions. This paper addresses this limitation through conformal prediction, a non-parametric forecasting technique seamlessly integrating with regression algorithms to produce prediction intervals at specified confidence levels. A crucial element in conformal prediction is the non-conformity score, traditionally based on absolute residual errors. In this work, we propose a novel approach, introducing data-dependent weights for computing non-conformity scores. This enhancement, considering the distances of training instances from the test sample, aims to improve overall algorithm performance. Empirical investigations across various real-world regression data sets, including scientific data, evaluate the efficiency and validity of prediction intervals from different uncertainty quantification methods. Results show that prediction intervals computed with data-dependent weights adapt to estimator uncertainty, offering more precise predictions in certain scenarios and appropriately conservative predictions in high uncertainty situations. Additionally, we compare predictive regions generated by conformal prediction with those from Gaussian Process Regression (GPR) for scientific data in structural engineering. To augment conformal prediction, we explore Conformalized Quantile Regression (CQR), a recent innovation combining conformal prediction with classical quantile regression, claiming full adaptability to heteroscedasticity. Our findings indicate that conformal prediction methods using data-dependent non-conformity scores achieve a 1% higher effective coverage level and a 15% reduction in prediction interval widths compared to other methods. The comparative analysis against GPR and CQR underscores the practical value of our approach in providing accurate prediction intervals in scientific and engineering domains.

Conformal Prediction Under Covariate Shift

Conformal Predictive Systems Under Covariate Shift

Distribution-Free Prediction Intervals Under Covariate Shift, With an Application to Causal Inference

Group-Weighted Conformal Prediction

Adaptive conformal inference under distribution shift

Conformal prediction with conditional guarantees

Conformal Prediction using Conditional Histograms

Conformal prediction with local weights: randomization enables local guarantees

Doubly robust calibration of prediction sets under covariate shift

Adapting Conformal Prediction to Distribution Shifts Without Labels

Conformal Prediction for Dose-Response Models with Continuous Treatments

Adaptive Conformal Prediction Intervals Using Data-Dependent Weights With Application to Seismic Response Prediction

Conformal prediction beyond exchangeability

Training-Conditional Coverage Bounds under Covariate Shift

Conformal prediction under feedback covariate shift for biomolecular design

Model-free selective inference under covariate shift via weighted conformal p-values

Conformal Prediction: A Gentle Introduction

Conformal Drug Property Prediction with Density Estimation under Covariate Shift

Conformal Prediction via Regression-as-Classification

Probabilistic Conformal Prediction with Approximate Conditional Validity

Efficient Conformal Prediction under Data Heterogeneity