Abstract:This study mainly concerns correction for measurement error using the meta-analysis of Fisher's z-transformed correlations. The disattenuation formula of Spearman (American Journal of Psychology, 15, 1904, 72) is used to correct for individual raw correlations in primary studies. The corrected raw correlations are then used to obtain the corrected z-transformed correlations. What remains little studied, however, is how to best correct for within-study sampling error variances of corrected z-transformed correlations. We focused on three within-study sampling error variance estimators corrected for measurement error that were proposed in earlier studies and is proposed in the current study: (1) the formula given by Hedges (Test validity, Lawrence Erlbaum, 1988) assuming a linear relationship between corrected and uncorrected z-transformed correlations (linear correction), (2) one derived by the first-order delta method based on the average of corrected z-transformed correlations (stabilized first-order correction), and (3) one derived by the second-order delta method based on the average of corrected z-transformed correlations (stabilized second-order correction). Via a simulation study, we compared performance of these estimators and the sampling error variance estimator uncorrected for measurement error in terms of estimation and inference accuracy of the mean correlation as well as the homogeneity test of effect sizes. In obtaining the corrected z-transformed correlations and within-study sampling error variances, coefficient alpha was used as a common reliability coefficient estimate. The results showed that in terms of the estimated mean correlation, sampling error variances with linear correction, the stabilized first-order and second-order corrections, and no correction performed similarly in general. Furthermore, in terms of the homogeneity test, given a relatively large average sample size and normal true scores, the stabilized first-order and second-order corrections had type I error rates that were generally controlled as well as or better than the other estimators. Overall, stabilized first-order and second-order corrections are recommended when true scores are normal, reliabilities are acceptable, the number of items per psychological scale is relatively large, and the average sample size is relatively large.

Using Generalizability Theory to Disattenuate Correlation Coefficients for Multiple Sources of Measurement Error

Practical Applications of Generalizability Theory for Designing, Evaluating, and Improving Psychological Assessments

Expanding G-Theory Models to Incorporate Congeneric Relationships: Illustrations Using the Big Five Inventory

Interrelationships between latent state-trait theory and generalizability theory within a structural equation modeling framework.

A Latent Trait Model for Multivariate Longitudinal Data With Two Sources of Measurement Error

Generalizability Theory as a Unifying Framework of Measurement Reliability in Adolescent Research

A Monte Carlo Study of Confidence Interval Methods for Generalizability Coefficient.

Meta-analysis of correlation coefficients: A cautionary tale on treating measurement error.

Using a linear mixed-effect model framework to estimate multivariate generalizability theory parameters in R

Analyzing Multivariate Generalizability Theory Designs within Structural Equation Modeling Frameworks

Reducing Attenuation Bias in Regression Analyses Involving Rating Scale Data via Psychometric Modeling

Beyond the factor indeterminacy problem using genome-wide association data

Correcting for Measurement Error under Meta-Analysis of Z-Transformed Correlations

Make Some Noise: Generating Data from Imperfect Factor Models

Benefits of Doing Generalizability Theory Analyses within Structural Equation Modeling Frameworks: Illustrations Using the Rosenberg Self-Esteem Scale

Corruption of the Pearson correlation coefficient by measurement error and its estimation, bias, and correction under different error models

Extending psychometric network analysis: Empirical evidence against g in favor of mutualism?

A general, simple, robust method to account for measurement error when analyzing data with an internal validation subsample

A true score imputation method to account for psychometric measurement error.

Reliability and External Validity of Personality Test Scores: The Role of Person and Item Error.

Comparing methods for assessing a difference in correlations with dependent groups, measurement error, nonnormality, and incomplete data.