Abstract:Handling missing data in clinical prognostic studies is an essential yet challenging task. This study aimed to provide a comprehensive assessment of the effectiveness and reliability of different machine learning (ML) imputation methods across various analytical perspectives. Specifically, it focused on three distinct classes of performance metrics used to evaluate ML imputation methods: post-imputation bias of regression estimates, post-imputation predictive accuracy, and substantive model-free metrics. As an illustration, we applied data from a real-world breast cancer survival study. This comprehensive approach aimed to provide a thorough assessment of the effectiveness and reliability of ML imputation methods across various analytical perspectives. A simulated dataset with 30% Missing At Random (MAR) values was used. A number of single imputation (SI) methods - specifically KNN, missMDA, CART, missForest, missRanger, missCforest - and multiple imputation (MI) methods - specifically miceCART and miceRF - were evaluated. The performance metrics used were Gower's distance, estimation bias, empirical standard error, coverage rate, length of confidence interval, predictive accuracy, proportion of falsely classified (PFC), normalized root mean squared error (NRMSE), AUC, and C-index scores. The analysis revealed that in terms of Gower's distance, CART and missForest were the most accurate, while missMDA and CART excelled for binary covariates; missForest and miceCART were superior for continuous covariates. When assessing bias and accuracy in regression estimates, miceCART and miceRF exhibited the least bias. Overall, the various imputation methods demonstrated greater efficiency than complete-case analysis (CCA), with MICE methods providing optimal confidence interval coverage. In terms of predictive accuracy for Cox models, missMDA and missForest had superior AUC and C-index scores. Despite offering better predictive accuracy, the study found that SI methods introduced more bias into the regression coefficients compared to MI methods. This study underlines the importance of selecting appropriate imputation methods based on study goals and data types in time-to-event research. The varying effectiveness of methods across the different performance metrics studied highlights the value of using advanced machine learning algorithms within a multiple imputation framework to enhance research integrity and the robustness of findings.

Multiple Imputation for Incomplete Data in Environmental Epidemiology Research

Multiple Imputation for Incomplete Data in Epidemiologic Studies

19 Incomplete Data in Epidemiology and Medical Statistics

Gaps in the usage and reporting of multiple imputation for incomplete data: findings from a scoping review of observational studies addressing causal questions

Evaluation of multiple imputation to address intended and unintended missing data in case-cohort studies with a binary endpoint

Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation

Multiple Imputation with Multivariate Imputation by Chained Equation (mice) Package

Multiple imputation methods for handling missing values in a longitudinal categorical variable with restrictions on transitions over time: a simulation study

Multiple imputation of missing data in multilevel ecological momentary assessments: an example using smoking cessation study data

A Comparison of Three Popular Methods for Handling Missing Data: Complete-Case Analysis, Inverse Probability Weighting, and Multiple Imputation

Multi-metric comparison of machine learning imputation methods with application to breast cancer survival

Multiple Imputation Approaches for Missing Time-to-Event Outcomes with Informative Censoring: Practical Considerations from a Simulation Study Based on Real Data

Population-calibrated multiple imputation for a binary/categorical covariate in categorical regression models

Analyses Using Multiple Imputation Need to Consider Missing Data in Auxiliary Variables

Multiple imputation for cause-specific Cox models: Assessing methods for estimation and prediction

CHOOSING APPROPRIATE IMPUTATION METHODS FOR MISSING DATA: A DECISION ALGORITHM ON METHODS FOR MISSING DATA

How handling missing data may impact conclusions: A comparison of six different imputation methods for categorical questionnaire data

A Comparative Study of Imputation Methods for Multivariate Ordinal Data

Imputation and Missing Indicators for handling missing data in the development and implementation of clinical prediction models: a simulation study

Multiple Imputation Methods for Missing Multilevel Ordinal Outcomes

Imputation methods for mixed datasets in bioarchaeology