Abstract:In numerous classification problems, class distribution is not balanced. For example, positive examples are rare in the fields of disease diagnosis and credit card fraud detection. General machine learning methods are known to be suboptimal for such imbalanced classification. One popular solution is to balance training data by oversampling the underrepresented (or undersampling the overrepresented) classes before applying machine learning algorithms. However, despite its popularity, the effectiveness of sampling has not been rigorously and comprehensively evaluated. This study assessed combinations of seven sampling methods and eight machine learning classifiers (56 varieties in total) using 31 datasets with varying degrees of imbalance. We used the areas under the precision-recall curve (AUPRC) and receiver operating characteristics curve (AUROC) as the performance measures. The AUPRC is known to be more informative for imbalanced classification than the AUROC. We observed that sampling significantly changed the performance of the classifier (paired t-tests P < 0.05) only for few cases (12.2% in AUPRC and 10.0% in AUROC). Surprisingly, sampling was more likely to reduce rather than improve the classification performance. Moreover, the adverse effects of sampling were more pronounced in AUPRC than in AUROC. Among the sampling methods, undersampling performed worse than others. Also, sampling was more effective for improving linear classifiers. Most importantly, we did not need sampling to obtain the optimal classifier for most of the 31 datasets. In addition, we found two interesting examples in which sampling significantly reduced AUPRC while significantly improving AUROC (paired t-tests P < 0.05). In conclusion, the applicability of sampling is limited because it could be ineffective or even harmful. Furthermore, the choice of the performance measure is crucial for decision making. Our results provide valuable insights into the effect and characteristics of sampling for imbalanced classification.

An Integrated Resampling Methods for Imbalanced Sporadic Temporal Data in EHRs

Imbalanced Data Sets Classification Method Based on Over-Sampling Technique

Imbalanced Data Classification Algorithm Based on Integrated Sampling and Ensemble Learning.

Resampling approach for imbalanced data classification based on class instance density per feature value intervals

Implications of resampling data to address the class imbalance problem (IRCIP): an evaluation of impact on performance between classification algorithms in medical data

A review on over-sampling techniques in classification of multi-class imbalanced datasets: insights for medical problems

Hospital readmission prediction with hybrid‐sampling and self‐paced balance learning

Minority Oversampling for Imbalanced Time Series Classification

Overlap-Based Undersampling Method for Classification of Imbalanced Medical Datasets

Bidirectional self-adaptive resampling in internet of things big data learning

Improved Oversampling Algorithm for Imbalanced Data Based on K-Nearest Neighbor and Interpolation Process Optimization

A self-inspected adaptive SMOTE algorithm (SASMOTE) for highly imbalanced data classification in healthcare

SADI: Similarity-Aware Diffusion Model-Based Imputation for Incomplete Temporal EHR Data

Adaptively weighted three-way decision oversampling: A cluster imbalanced-ratio based approach

Selecting the Suitable Resampling Strategy for Imbalanced Data Classification Regarding Dataset Properties. An Approach Based on Association Models

Adaptive Subspace Sampling for Class Imbalance Processing-Some clarifications, algorithm, and further investigation including applications to Brain Computer Interface

An empirical evaluation of sampling methods for the classification of imbalanced data

Gaussian Distribution Based Oversampling for Imbalanced Data Classification

Dynamic self-paced sampling ensemble for highly imbalanced and class-overlapped data classification

An adaptive Laplacian weight random forest imputation for imbalance and mixed-type data

Selecting the suitable resampling strategy for imbalanced data classification regarding dataset properties