Abstract:The correlation coefficient is one of the essential statistical techniques used to discover relationships among variables. Various techniques can quantify correlation, such as Pearson's, Spearman's, and Kendall's correlation coefficients, depending on the data type. As with any use of data, missing data will impact the availability of data, reducing it and potentially affecting the results. Furthermore, the removal of missing-value data from the study when using complete case analysis or available case analysis may result in selection biases. In this paper, we investigate the impact of missing data on the correlation coefficient value by calculating the difference between the correlation coefficient of the original complete dataset and that of a dataset with missing data. Two deletion strategies (Listwise and Pairwise) and three imputation strategies (Mean, k-Nearest Neighbors (k-NN), and Expectation-Maximization) were used to prepare the data before calculating the correlation coefficient. Unique correlation coefficient values were created by converting unique values to a one-dimensional array, and RMSE metrics were used to evaluate the experiments. Eight UCI and Kaggle datasets with different sizes and numbers of attributes were used in this study. The experiment results demonstrate that the Pairwise strategy and k-NN give good results on the correlation coefficient, respectively, when the missing rate is moderate or less. Pairwise uses all the available values and discards only the missing values of the related attribute, while k-NN fills the missing values with new values that produce correlation coefficient values close to the actual values.

Sample Correlation Coefficients Based on Survey Data under Regression Imputation

Compromised imputation based mean estimators using robust quantile regression

Optimized estimation for population mean using conventional and non-conventional measures under the joint influence of measurement error and non-response

Regression with an Imputed Dependent Variable

Joint Mean and Correlation Regression Models for Multivariate Data

Correlation visualization under missing values: a comparison between imputation and direct parameter estimation methods

Iteratively Reweighted Least Squares Method for Estimating Polyserial and Polychoric Correlation Coefficients

Network Imputation for a Spatial Autoregression Model with Incomplete Data

Impact of Missing Data on Correlation Coefficient Values: Deletion and Imputation Methods for Data Preparation

On the Relation between Prediction and Imputation Accuracy under Missing Covariates

Nonparametric Imputation of Missing Values for Estimating Equation Based Inference

Robust Estimating Equations and Bias Correction of Correlation Parameters for Longitudinal Data

Projective Resampling Imputation Mean Estimation Method for Missing Covariates Problem

Regression models of Pearson correlation coefficient

Sampling Importance Resampling Algorithm with Nonignorable Missing Response Variable Based on Smoothed Quantile Regression

Imputations for High Missing Rate Data in Covariates Via Semi-supervised Learning Approach

Robust propensity score weighting estimation under missing at random

Modified correlated measurement errors model for estimation of population mean utilizing auxiliary information

Exact balanced random imputation for sample survey data

Statistical inference on seemingly unrelated non-parametric regression models with serially correlated errors