Abstract:Focus on predictive algorithm and its performance evaluation is extensively covered in most research studies to determine best or appropriate predictive model with Optimum prediction solution indicated by prediction accuracy score, precision, recall, f1score etc. Prediction accuracy score from performance evaluation has been used extensively as the main determining metric for performance recommendation. It is one of the most widely used metric for identifying optimal prediction solution irrespective of dataset class distribution context or nature of dataset and output class distribution between the minority and majority variables. The key research question however is the impact of class inequality on prediction accuracy score in such datasets with output class distribution imbalance as compared to balanced accuracy score in the determination of model performance in healthcare and other real-world application systems. Answering this question requires an appraisal of current state of knowledge in both prediction accuracy score and balanced accuracy score use in real-world applications where there is unequal class distribution. Review of related works that highlight the use of imbalanced class distribution datasets with evaluation metrics will assist in contextualizing this systematic review. The incidence of unequal class distribution in real-world applications such as healthcare and other non-medical settings continue to receive attention due to machine learning technique challenges with minority class contribution in datasets with imbalanced class distribution. Challenges such as discounting minority class contribution which may be the subject of interest. Predictive modeling evaluation of such datasets with prediction accuracy score which does not take into account dataset class distribution variation could create an erroneous impression of a supposedly high performing machine learning technique as it discounts minority class contribution. Estimating predictive model performance with balanced accuracy score that incorporates other important metrics such as true positives, true positive rates, true negatives, true negative rates, false positive, false positive rates, false negatives and false negative rates could help assess machine learning model performance more adequately and accurately to determine appropriate model performance.

Fair evaluation of classifier predictive performance based on binary confusion matrix

Empirical analysis of performance assessment for imbalanced classification

Unified Performance Measure for Binary Classification Problems

BenchMetrics: a systematic benchmarking method for binary classification performance metrics

Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems

Balancing the Scales: A Comprehensive Study on Tackling Class Imbalance in Binary Classification

Comparative study of quality estimation of binary classification

Empirical study of Machine Learning Classifier Evaluation Metrics behavior in Massively Imbalanced and Noisy data

Fairness and Unfairness in Binary and Multiclass Classification: Quantifying, Calculating, and Bounding

Comparing alternative classifiers for database marketing: The case of imbalanced datasets

The Effect of Balancing Methods on Model Behavior in Imbalanced Classification Problems

Mind your prevalence!

An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classification

Robust performance metrics for imbalanced classification problems

A measure oriented training scheme for imbalanced classification problems

A study on cost behaviors of binary classification measures in class-imbalanced problems

Hierarchical confusion matrix for classification performance evaluation

Evaluation metrics and dimensional reduction for binary classification algorithms: a case study on bankruptcy prediction

Binary Classification: Counterbalancing Class Imbalance by Applying Regression Models in Combination with One-Sided Label Shifts

A refined approach for evaluating small datasets via binary classification using machine learning