A Simulation Study on the Detection of Measurement Invariance and Its Influential Factors for Ordinal Categorical Data
Li Chong,Liu Hongyun
DOI: https://doi.org/10.16719/j.cnki.1671-6981.2011.06.037
2011-01-01
Abstract:Measurement equivalence/invariance(ME/I) is obtained when the relations between observed test scores and the latent trait measured by the test are identical across subpopulations.In particular,individuals with equal standings on the latent trait,but sampled from different subpopulations,should have the same expected observed score.In social sciences,researchers collect and analyze ordinal data frequently,such as data obtained from Likert Scale.While researchers often treat ordinal data as continuous,the ordinal measurement is discrete in nature.Then the x~2 statistic and standard errors of parameter estimation tend to exhibit bias.As a result,the results of the nested model test for exploring ME/I based on the normal theory should be affected.This simulation study uses simulated data with known properties to assess the effect of parameters recovery,difference between estimated value and known population parameters value,and empirical power in various total sample sizes,the ratio of the reference group to the focal group size,the difference of parameters between two groups,and test length. The present study introduced a new method of model construction(latent response variable model,LRV) and parameters estimation (robust weighted least squares estimator with a diagonal weight matrix,WLSMV) for ordinal categorical data,with the method of exploring ME/I based on this method(DIFFTEST).Two levels of test length were examined:a short test(10 items) and a moderate test(20 items).To control the number of items whose ME/I were violated,the ME/I of 5 items in these two conditions were simulated to be violated.Three levels of overall sample size(500,1000,and 2000),as well as two different sample size ratios of reference group to focal group members(5/5 and 8/2) were considered.Another condition of interest in this study was the difference of threshold between the two groups.Four levels of thresholds difference were examined in this study:0,.2,.4,.6.which reflected the violation degree,from zero,low,medium to high,respectively.Especially,the condition of zero threshold difference was designed to study the typeⅠerror,examining false detection of DIFFTEST for ordinal categorical data. The results indicated that the parameters recovered well when WLSMV method was used.The type I error of DIFFTEST consistently adhered closely to the nominal alpha level of 0.05.The power of DIFFTEST increased when the sample size increased,or the threshold difference became greater,or reference and focal group size became equal,which indicated applied researchers should make efforts to keep both group sizes as large as possible rather than relying on one very large group.With the controlled number of items whose ME/ I was violated,the power of DIFFTEST became lower when the test length became longer.