Abstract:In the realm of medical datasets, particularly when considering diabetes, the occurrence of data incompleteness is a prevalent issue. Unveiling valuable patterns through medical data analysis is crucial for early and precise medical predictions. However, the quality of data and the proper handling of missing data hold significant significance. To address this challenge, imputation stands as a robust approach. The main goal of this paper aims to provide a comprehensive investigation into the effects brought about by Machine Learning (ML) based imputation techniques, specifically K Nearest Neighbor Imputation (KNNI), Multiple Imputation by Chained Equations (MICE), and MissForest. Results of all three techniques are compared with the complete dataset for five missing rates (10% to 50%), and evaluated using four categories of evaluation criteria i.e. (1) model performance, (2) imputation error rate (Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Coefficient of Determination (R^2) values), (3) Pearson correlation analysis and, (4) model selection basis (Bayesian information criterion (BIC), Akaike information criterion (AIC), values). Model performance includes accuracy, precision, recall, F1 score, and Matthews Correlation Coefficient (Mcoff) score of four ML classifiers viz. (a) Random Forest (RF), (b) Support vector machine (SVM), (c) AdaBoost, (d) XGBoost (XGB). For all missing rate cases, the MissForest technique is better than the KNNI and MICE in accuracy and Mcoff in 80% of cases, precision in 40% of cases, recall in 60% of cases, F1 score, MAE, RMSE, R^2 in 100% of cases, AIC in 80% of cases, and BIC values in 100% of cases. Also, the correlation analysis confirms that the MissForest imputation preserves association between the variables, like the complete dataset. Overall, our findings suggest that MissForest is a better machine learning-based imputation technique for handling missing data in diabetes research.

iDMI: A novel technique for missing value imputation using a decision tree and expectation-maximization algorithm

Evaluation of imputation techniques with varying percentage of missing data

SICE: an improved missing data imputation technique

A Robust Missing Value Imputation Method MifImpute For Incomplete Molecular Descriptor Data And Comparative Analysis With Other Missing Value Imputation Methods

Data Envelopment Analysis with Missing Data: a Multiple Imputation Approach

Unleashing the Potential of Diffusion Models for Incomplete Data Imputation

Missing Data Imputation: Focusing on Single Imputation.

Missing Data Exploration: Highlighting Graphical Presentation of Missing Pattern.

An Intelligent Missing Data Imputation Techniques: A Review

Relational Data Imputation with Quality Guarantee.

CHOOSING APPROPRIATE IMPUTATION METHODS FOR MISSING DATA: A DECISION ALGORITHM ON METHODS FOR MISSING DATA

Impact of machine learning-based imputation techniques on medical datasets- a comparative analysis

On-Line Imputation For Missing Values

Missing Value Estimation using Clustering and Deep Learning within Multiple Imputation Framework

Examination of Reliability of Missing Value Recovery in Data Mining

Missing value imputation using unsupervised machine learning techniques

Efficient missing data imputation for supervised learning.

Information-decomposition-model-based Missing Value Estimation for Not Missing at Random Dataset

Semi-parametric Optimization for Missing Data Imputation

Imputation using information fusion technique for sensor generated incomplete data with high missing gap

In-Database Data Imputation