Abstract:The classification of imbalanced data has presented a significant challenge for most well-known classification algorithms that were often designed for data with relatively balanced class distributions. Nevertheless skewed class distribution is a common feature in real world problems. It is especially prevalent in certain application domains with great need for machine learning and better predictive analysis such as disease diagnosis, fraud detection, bankruptcy prediction, and suspect identification. In this paper, we propose a novel tree-based algorithm based on the area under the precision-recall curve (AUPRC) for variable selection in the classification context. Our algorithm, named as the "Precision-Recall Curve classification tree", or simply the "PRC classification tree" modifies two crucial stages in tree building. The first stage is to maximize the area under the precision-recall curve in node variable selection. The second stage is to maximize the harmonic mean of recall and precision (F-measure) for threshold selection. We found the proposed PRC classification tree, and its subsequent extension, the PRC random forest, work well especially for class-imbalanced data sets. We have demonstrated that our methods outperform their classic counterparts, the usual CART and random forest for both synthetic and real data. Furthermore, the ROC classification tree proposed by our group previously has shown good performance in imbalanced data. The combination of them, the PRC-ROC tree, also shows great promise in identifying the minority class.

Binormal Precision–Recall Curves for Optimal Classification of Imbalanced Data

The receiver operating characteristic curve accurately assesses imbalanced datasets

The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets

Tuning model parameters in class‐imbalanced learning with precision‐recall curve

Precision-Recall Curve (PRC) Classification Trees

”Proper” binormal ROC curves: theory and maximum-likelihood estimation

The “Proper” binormal model: Parametric receiver operating characteristic curve estimation with degenerate data

Precision and Recall Reject Curves for Classification

Optimal ROC-Based Classification and Performance Analysis under Bayesian Uncertainty Models

Classification performance assessment for imbalanced multiclass data

A placement-value based approach to concave ROC analysis

The MCC-F1 curve: a performance evaluation technique for binary classification

Implications of Imbalanced Datasets for Empirical ROC-AUC Estimation in Binary Classification Tasks

Population and Empirical PR Curves for Assessment of Ranking Algorithms

Finding the Best Classification Threshold in Imbalanced Classification

Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functions

Bayesian ROC surface estimation under verification bias

Nonparametric receiver operating characteristic curve analysis with an imperfect gold standard

Small-sample precision of ROC-related estimates

Area under Precision-Recall Curves for Weighted and Unweighted Data