Abstract:Abstract Objective Methods to correct class imbalance (imbalance between the frequency of outcome events and nonevents) are receiving increasing interest for developing prediction models. We examined the effect of imbalance correction on the performance of logistic regression models. Material and Methods Prediction models were developed using standard and penalized (ridge) logistic regression under 4 methods to address class imbalance: no correction, random undersampling, random oversampling, and SMOTE. Model performance was evaluated in terms of discrimination, calibration, and classification. Using Monte Carlo simulations, we studied the impact of training set size, number of predictors, and the outcome event fraction. A case study on prediction modeling for ovarian cancer diagnosis is presented. Results The use of random undersampling, random oversampling, or SMOTE yielded poorly calibrated models: the probability to belong to the minority class was strongly overestimated. These methods did not result in higher areas under the ROC curve when compared with models developed without correction for class imbalance. Although imbalance correction improved the balance between sensitivity and specificity, similar results were obtained by shifting the probability threshold instead. Discussion Imbalance correction led to models with strong miscalibration without better ability to distinguish between patients with and without the outcome event. The inaccurate probability estimates reduce the clinical utility of the model, because decisions about treatment are ill-informed. Conclusion Outcome imbalance is not a problem in itself, imbalance correction may even worsen model performance.

Logistic Regression Bias Correction for Large Scale Data with Rare Events

Logistic Regression in Rare Events Data

Distributed Logistic Regression for Massive Data with Rare Events

Bias Correction in Machine Learning-based Classification of Rare Events

Causally Regularized Learning with Agnostic Data Selection Bias

Mastering Rare Event Analysis: Optimal Subsample Size in Logistic and Cox Regressions

Rectify the Regression Bias in Long-Tailed Object Detection

Robust and efficient subsampling algorithms for massive data logistic regression

Avoiding spurious correlations via logit correction

Supporting Regularized Logistic Regression Privately and Efficiently

Affine combination‐based over‐sampling for imbalanced regression

A Generalized Doubly Robust Learning Framework for Debiasing Post-Click Conversion Rate Prediction

Robust estimation and bias-corrected empirical likelihood in generalized linear models with right censored data

Logistic regression analysis with standardized markers

Logistic Regression with Misclassification in Binary Outcome Variables: a Method and Software

A Descriptive Study of Variable Discretization and Cost-Sensitive Logistic Regression on Imbalanced Credit Data

Robust adaptive LASSO in high-dimensional logistic regression

Fixing imbalanced binary classification: An asymmetric Bayesian learning approach

Large-scale Causal Approaches to Debiasing Post-click Conversion Rate Estimation with Multi-task Learning

Efficient Online Learning for Large-Scale Sparse Kernel Logistic Regression

The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regression