Abstract:Focus on predictive algorithm and its performance evaluation is extensively covered in most research studies to determine best or appropriate predictive model with Optimum prediction solution indicated by prediction accuracy score, precision, recall, f1score etc. Prediction accuracy score from performance evaluation has been used extensively as the main determining metric for performance recommendation. It is one of the most widely used metric for identifying optimal prediction solution irrespective of dataset class distribution context or nature of dataset and output class distribution between the minority and majority variables. The key research question however is the impact of class inequality on prediction accuracy score in such datasets with output class distribution imbalance as compared to balanced accuracy score in the determination of model performance in healthcare and other real-world application systems. Answering this question requires an appraisal of current state of knowledge in both prediction accuracy score and balanced accuracy score use in real-world applications where there is unequal class distribution. Review of related works that highlight the use of imbalanced class distribution datasets with evaluation metrics will assist in contextualizing this systematic review. The incidence of unequal class distribution in real-world applications such as healthcare and other non-medical settings continue to receive attention due to machine learning technique challenges with minority class contribution in datasets with imbalanced class distribution. Challenges such as discounting minority class contribution which may be the subject of interest. Predictive modeling evaluation of such datasets with prediction accuracy score which does not take into account dataset class distribution variation could create an erroneous impression of a supposedly high performing machine learning technique as it discounts minority class contribution. Estimating predictive model performance with balanced accuracy score that incorporates other important metrics such as true positives, true positive rates, true negatives, true negative rates, false positive, false positive rates, false negatives and false negative rates could help assess machine learning model performance more adequately and accurately to determine appropriate model performance.

Strengthening Machine Learning Reproducibility for Image Classification

A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifiers

A Holistic Assessment of the Reliability of Machine Learning Systems

Towards Inferential Reproducibility of Machine Learning Research

Reproducibility of Machine Learning: Terminology, Recommendations and Open Issues

A Classfication Method For Imbalance Data Set Based on Kernel SMOTE

A Large-Scale Empirical Study on Improving the Fairness of Image Classification Models

Machine Learning Evaluation Metric Discrepancies across Programming Languages and Their Components: Need for Standardization

Improving the repeatability of deep learning models with Monte Carlo dropout

Empirical study of Machine Learning Classifier Evaluation Metrics behavior in Massively Imbalanced and Noisy data

Evaluation Gaps in Machine Learning Practice

Empirical analysis of performance assessment for imbalanced classification

A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification

A critical moment in machine learning in medicine: on reproducible and interpretable learning

Evaluation metrics and statistical tests for machine learning

Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems

On Model Evaluation under Non-constant Class Imbalance

Uncertainty-aware Evaluation of Machine Learning Performance in binary Classification Tasks

Bias Assessment and Data Drift Detection in Medical Image Analysis: A Survey

Measuring Class-Imbalance Sensitivity of Deterministic Performance Evaluation Metrics

Questionable practices in machine learning