Abstract:Machine learning often lacks transparent performance indicators, especially in generating point predictions. This paper addresses this limitation through conformal prediction, a non-parametric forecasting technique seamlessly integrating with regression algorithms to produce prediction intervals at specified confidence levels. A crucial element in conformal prediction is the non-conformity score, traditionally based on absolute residual errors. In this work, we propose a novel approach, introducing data-dependent weights for computing non-conformity scores. This enhancement, considering the distances of training instances from the test sample, aims to improve overall algorithm performance. Empirical investigations across various real-world regression data sets, including scientific data, evaluate the efficiency and validity of prediction intervals from different uncertainty quantification methods. Results show that prediction intervals computed with data-dependent weights adapt to estimator uncertainty, offering more precise predictions in certain scenarios and appropriately conservative predictions in high uncertainty situations. Additionally, we compare predictive regions generated by conformal prediction with those from Gaussian Process Regression (GPR) for scientific data in structural engineering. To augment conformal prediction, we explore Conformalized Quantile Regression (CQR), a recent innovation combining conformal prediction with classical quantile regression, claiming full adaptability to heteroscedasticity. Our findings indicate that conformal prediction methods using data-dependent non-conformity scores achieve a 1% higher effective coverage level and a 15% reduction in prediction interval widths compared to other methods. The comparative analysis against GPR and CQR underscores the practical value of our approach in providing accurate prediction intervals in scientific and engineering domains.

Deep Conformal Supervision: Leveraging Intermediate Features for Robust Uncertainty Quantification

Empirical Validation of Conformal Prediction for Trustworthy Skin Lesions Classification

Quantifying Deep Learning Model Uncertainty in Conformal Prediction

Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets

Fair Conformal Predictors for Applications in Medical Imaging

Toward Clinically Trustworthy Deep Learning: Applying Conformal Prediction to Intracranial Hemorrhage Detection

DOA Uncertainty Quantification With Conformal Prediction

Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework

Conformal Risk Control for Pulmonary Nodule Detection

Adaptive Uncertainty Quantification for Generative AI

Conformalized-DeepONet: A Distribution-Free Framework for Uncertainty Quantification in Deep Operator Networks

A Comparative Study of Confidence Calibration in Deep Learning: From Computer Vision to Medical Imaging

Conformal Prediction on Quantifying Uncertainty of Dynamic Systems

Uncertainty Quantification of Surrogate Models using Conformal Prediction

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling

Onboard Out-of-Calibration Detection of Deep Learning Models using Conformal Prediction

Uncertainty Aware Training to Improve Deep Learning Model Calibration for Classification of Cardiac MR Images

Improving Uncertainty Quantification of Deep Classifiers via Neighborhood Conformal Prediction: Novel Algorithm and Theoretical Analysis

Trustworthy Classification through Rank-Based Conformal Prediction Sets

Adaptive Conformal Prediction Intervals Using Data-Dependent Weights With Application to Seismic Response Prediction

Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction