Abstract:Medical imaging classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, which prevents their deployment in medical clinics. We present an algorithm that can modify any classifier to produce a prediction set containing the true label with a user-specified probability, such as 90%. We train a network to predict an instance-based version of the Conformal Prediction threshold. The threshold is then conformalized to ensure the required coverage. We applied the proposed algorithm to several standard medical imaging classification datasets. The experimental results demonstrate that our method outperforms current approaches in terms of smaller average size of the prediction set while maintaining the desired coverage.

What problem does this paper attempt to address?

The paper attempts to address the problem of how to quantify the uncertainty of classifiers in medical image classification tasks and generate a prediction set that includes the true label with high probability (e.g., 90%). Specifically, although existing medical image classifiers can achieve high prediction accuracy, there are still challenges in quantifying their uncertainty, which limits their application in clinical practice. To tackle this challenge, the authors propose a new algorithm that can modify any classifier to generate a prediction set that includes the true label with a user-specified probability (e.g., 90%). Compared to existing methods, this algorithm can generate smaller average prediction sets while maintaining the required coverage. ### Main Contributions: 1. **Proposed a new algorithm**: By training a neural network to predict personalized thresholds for each sample, thereby generating prediction sets. 2. **Improved the efficiency of prediction sets**: Experimental results show that this method significantly outperforms existing state-of-the-art methods on multiple standard medical image classification datasets, especially in generating smaller prediction sets. 3. **Applicable to medical diagnosis**: Prediction sets are particularly valuable in medical diagnosis as they can help doctors rule out multiple possibilities and quickly refer patients to the appropriate specialists. ### Method Overview: - **Conformal Prediction (CP)**: A non-parametric calibration method used to construct prediction sets, ensuring that the probability of the correct category is at least 1-α. - **Adaptive Prediction Sets (APS)** and **Regularized Adaptive Prediction Sets (RAPS)**: Existing CP variants used to generate prediction sets. - **Conformalized Prediction Set Network (CPSN)**: The algorithm proposed in this paper, which trains a neural network to predict personalized thresholds for each sample and corrects them to ensure the required coverage. ### Experimental Results: - **Datasets**: OrganAMNIST and TissuMNIST, containing abdominal CT images and human kidney cortex cell images, respectively. - **Evaluation Metrics**: The size of the prediction set (the smaller, the better) and the coverage (the proportion of prediction sets that include the true label). - **Experimental Results**: CPSN can generate smaller and lower variance prediction sets under all settings while meeting the coverage requirements. In summary, this paper proposes a new algorithm to address the challenge of quantifying uncertainty in medical image classification tasks, providing a more reliable and efficient solution for clinical applications.

A conformalized learning of a prediction set with applications to medical imaging classification

Fair Conformal Predictors for Applications in Medical Imaging

A Conformal Prediction Score that is Robust to Label Noise

Using random forest for reliable classification and cost-sensitive learning for medical diagnosis

Empirical Validation of Conformal Prediction for Trustworthy Skin Lesions Classification

Conformal confidence sets for biomedical image segmentation

Uncertainty sets for image classifiers using conformal prediction

Conformal Prediction: A Gentle Introduction

RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classification

Three Applications of Conformal Prediction for Rating Breast Density in Mammography

Class-Conditional Conformal Prediction with Many Classes

Trustworthy Classification through Rank-Based Conformal Prediction Sets

Optimized conformal classification using gradient descent approximation

Conformal Prediction for Deep Classifier via Label Ranking

Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage

Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets

Conformal Prediction with Learned Features

Conformal Prediction Sets Improve Human Decision Making

Metric-guided Image Reconstruction Bounds via Conformal Prediction

Reliable machine learning models in genomic medicine using conformal prediction

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling