Abstract:Background Most machine-learning classifiers output label predictions for new instances without indicating how reliable the predictions are. The applicability of these classifiers is limited in critical domains where incorrect predictions have serious consequences, like medical diagnosis. Further, the default assumption of equal misclassification costs is most likely violated in medical diagnosis. Results In this paper, we present a modified random forest classifier which is incorporated into the conformal predictor scheme. A conformal predictor is a transductive learning scheme, using Kolmogorov complexity to test the randomness of a particular sample with respect to the training sets. Our method show well-calibrated property that the performance can be set prior to classification and the accurate rate is exactly equal to the predefined confidence level. Further, to address the cost sensitive problem, we extend our method to a label-conditional predictor which takes into account different costs for misclassifications in different class and allows different confidence level to be specified for each class. Intensive experiments on benchmark datasets and real world applications show the resultant classifier is well-calibrated and able to control the specific risk of different class. Conclusion The method of using RF outlier measure to design a nonconformity measure benefits the resultant predictor. Further, a label-conditional classifier is developed and turn to be an alternative approach to the cost sensitive learning problem that relies on label-wise predefined confidence level. The target of minimizing the risk of misclassification is achieved by specifying the different confidence level for different class.

Calibrating probabilistic predictions of quantile regression forests with conformal predictive systems

Adaptive Conformal Prediction by Reweighting Nonconformity Score

A Conformal Predictive System for Distribution Regression with Random Features

Conformalized Unconditional Quantile Regression

A Fast Conformal Predictive System with Regularized Extreme Learning Machine.

Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration

Conformal Prediction: A Data Perspective

Robust Conformal Prediction under Distribution Shift via Physics-Informed Structural Causal Model

Using random forest for reliable classification and cost-sensitive learning for medical diagnosis

Optimizing Probabilistic Conformal Prediction with Vectorized Non-Conformity Scores

Nested conformal prediction and quantile out-of-bag ensemble methods

Conformalized quantile regression

Probabilistic Conformal Prediction with Approximate Conditional Validity

Global Censored Quantile Random Forest

Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting

Supplementary Material for Conformalized Quantile Regression

Nonparametric Quantile Regression: Non-Crossing Constraints and Conformal Prediction

Conformalized Interval Arithmetic with Symmetric Calibration

Density-Calibrated Conformal Quantile Regression

Probabilistic prediction with locally weighted jackknife predictive system

Quantile Regression using Random Forest Proximities