Abstract:BackgroundIn modern biomedical research of complex diseases, a large number of demographic and clinical variables, herein called phenomic data, are often collected and missing values (MVs) are inevitable in the data collection process. Since many downstream statistical and bioinformatics methods require complete data matrix, imputation is a common and practical solution. In high-throughput experiments such as microarray experiments, continuous intensities are measured and many mature missing value imputation methods have been developed and widely applied. Numerous methods for missing data imputation of microarray data have been developed. Large phenomic data, however, contain continuous, nominal, binary and ordinal data types, which void application of most methods. Though several methods have been developed in the past few years, not a single complete guideline is proposed with respect to phenomic missing data imputation.ResultsIn this paper, we investigated existing imputation methods for phenomic data, proposed a self-training selection (STS) scheme to select the best imputation method and provide a practical guideline for general applications. We introduced a novel concept of "imputability measure" (IM) to identify missing values that are fundamentally inadequate to impute. In addition, we also developed four variations of K-nearest-neighbor (KNN) methods and compared with two existing methods, multivariate imputation by chained equations (MICE) and missForest. The four variations are imputation by variables (KNN-V), by subjects (KNN-S), their weighted hybrid (KNN-H) and an adaptively weighted hybrid (KNN-A). We performed simulations and applied different imputation methods and the STS scheme to three lung disease phenomic datasets to evaluate the methods. An R package "phenomeImpute" is made publicly available.ConclusionsSimulations and applications to real datasets showed that MICE often did not perform well; KNN-A, KNN-H and random forest were among the top performers although no method universally performed the best. Imputation of missing values with low imputability measures increased imputation errors greatly and could potentially deteriorate downstream analyses. The STS scheme was accurate in selecting the optimal method by evaluating methods in a second layer of missingness simulation. All source files for the simulation and the real data analyses are available on the author's publication website.

Imputation of missing values in DNA microarray gene expression data

An Iterative Locally Auto-Weighted Least Squares Method for Microarray Missing Value Estimation

Usage of Clustering and Weighted Nearest Neighbors for Efficient Missing Data Imputation of Microarray Gene Expression Dataset

An imputation approach for oligonucleotide microarrays

Missing values estimation in microarray data with partial least squares regression

Missing Value Estimation Algorithms on Cluster and Representativeness Preservation of Gene Expression Microarray Data

Missing Value Estimation for Microarray Data by Bayesian Principal Component Analysis and Iterative Local Least Squares

Evaluations on Several Imputation Approaches of Integrated Omics Data

Improved k-nearest neighbors approach for incomplete and contaminated gene expression datasets

Missing value imputation in high-dimensional phenomic data: imputable or not, and how?

Accurate and interpretable gene expression imputation on scRNA-seq data using IGSimpute

Fast matrix completion in epigenetic methylation studies with informative covariates

Missing Value Imputation With Low-Rank Matrix Completion in Single-Cell RNA-Seq Data by Considering Cell Heterogeneity

Missing Data Imputation: Focusing on Single Imputation.

Imputing Missing Values for Genetic Interaction Data.

Nonparametric Imputation of Missing Values for Estimating Equation Based Inference

A joint optimization framework integrated with biological knowledge for clustering incomplete gene expression data

scCGImpute: An Imputation Method for Single-Cell RNA Sequencing Data Based on Similarities between Cells and Relationships among Genes

Imputation of missing values in lipidomic datasets

Imputing not available values in single-cell DNA methylation data using the median is straightforward and effective

Evaluation of different approaches for missing data imputation on features associated to genomic data