Abstract:Objectives: The development of clinical prediction models is often impeded by the occurrence of missing values in the predictors. Various methods for imputing missing values before modeling have been proposed. Some of them are based on variants of multiple imputations by chained equations, while others are based on single imputation. These methods may include elements of flexible modeling or machine learning algorithms, and for some of them user-friendly software packages are available. The aim of this study was to investigate by simulation if some of these methods consistently outperform others in performance measures of clinical prediction models. Study design and setting: We simulated development and validation cohorts by mimicking observed distributions of predictors and outcome variable of a real data set. In the development cohorts, missing predictor values were created in 36 scenarios defined by the missingness mechanism and proportion of noncomplete cases. We applied three imputation algorithms that were available in R software (R Foundation for Statistical Computing, Vienna, Austria): mice, aregImpute, and missForest. These algorithms differed in their use of linear or flexible models, or random forests, the way of sampling from the predictive posterior distribution, and the generation of a single or multiple imputed data set. For multiple imputation, we also investigated the impact of the number of imputations. Logistic regression models were fitted with the simulated development cohorts before (full data analysis) and after missing value generation (complete case analysis), and with the imputed data. Prognostic model performance was measured by the scaled Brier score, c-statistic, calibration intercept and slope, and by the mean absolute prediction error evaluated in validation cohorts without missing values. Performance of full data analysis was considered as ideal. Results: None of the imputation methods achieved the model's predictive accuracy that would be obtained in case of no missingness. In general, complete case analysis yielded the worst performance, and deviation from ideal performance increased with increasing percentage of missingness and decreasing sample size. Across all scenarios and performance measures, aregImpute and mice, both with 100 imputations, resulted in highest predictive accuracy. Surprisingly, aregImpute outperformed full data analysis in achieving calibration slopes very close to one across all scenarios and outcome models. The increase of mice's performance with 100 compared to five imputations was only marginal. The differences between the imputation methods decreased with increasing sample sizes and decreasing proportion of noncomplete cases. Conclusion: In our simulation study, model calibration was more affected by the choice of the imputation method than model discrimination. While differences in model performance after using imputation methods were generally small, multiple imputation methods as mice and aregImpute that can handle linear or nonlinear associations between predictors and outcome are an attractive and reliable choice in most situations.

Dirichlet process mixture models to impute missing predictor data in counterfactual prediction models: an application to predict optimal type 2 diabetes therapy

Missing Data Imputation for a Multivariate Outcome of Mixed Variable Types

Personalized Imputation in metric spaces via conformal prediction: Applications in Predicting Diabetes Development with Continuous Glucose Monitoring Information

Causal Inference and Counterfactual Prediction in Machine Learning for Actionable Healthcare

An Imputation model by Dirichlet Process Mixture of Elliptical Copulas for Data of Mixed Type

Nonparametric Statistical Inference and Imputation for Incomplete Categorical Data

Imputation and Missing Indicators for handling missing data in the development and implementation of clinical prediction models: a simulation study

Outcome identification in electronic health records using predictions from an enriched Dirichlet process mixture

The performance of prognostic models depended on the choice of missing value imputation algorithm: a simulation study

Use of Multiple Imputation Models in Medical Device Trials

Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study

An Efficient Multiple Imputation Algorithm for Control-Based and Delta-Adjusted Pattern Mixture Models using SAS

To Impute or not to Impute? Missing Data in Treatment Effect Estimation

Handling Missing Data in Growth Mixture Models

A Copula-based Imputation Model for Missing Data of Mixed Type in Multilevel Data Sets

A Bayesian two-step multiple imputation approach based on mixed models for the missing in EMA data

Robustness of Multiple Imputation Methods for Missing Risk Factor Data from Electronic Medical Records for Observational Studies

Estimating hypothetical estimands with causal inference and missing data estimators in a diabetes trial

Additive mixed models with approximate Dirichlet process mixtures: the EM approach

Variable selection with missing data in both covariates and outcomes: Imputation and machine learning

Imputation of Counterfactual Outcomes when the Errors are Predictable