Abstract:Differential item functioning (DIF) is an important issue in large scale standardized testing. DIF refers to the unexpected difference in item performances among groups of equally proficient examinees, usually classified by ethnicity or gender. Its presence could seriously affect the validity of inferences drawn from a test. Various statistical methods have been proposed to detect and estimate DIF. This dissertation addresses DIF analysis in the context of computerized adaptive testing (CAT), whose item selection algorithm adapts to the ability level of each individual examinee. In a CAT, a DIF item may be more consequential and more detrimental be cause fewer items are administered in a CAT than in a traditional paper-and-pencil test and because the remaining sequence of items presented to examinees depends in part on their responses to the DIF item. Consequently, an efficient, stable and flexible method to detect and estimate CAT DIF becomes necessary and increasingly important. We propose simultaneous implementations of online calibration and DIF testing. The idea is to perform online calibration of an item of interest separately in the focal and reference groups. Under any specific parametric IRT model, we can use the (online) estimated latent traits as covariates and fit a nonlinear regression model to each of the two groups. Because of the use of the estimated, not the true t, the regression fit has to adjust for the covariate “measurement errors”. It turns out that this situation fits nicely into the framework of nonlinear error-in-variable modelling, which has been extensively studied in statistical literature. We develop two bias-correction methods using asymptotic expansion and conditional score theory. After correcting the bias caused by measurement error, one can perform a significance test to detect DIF with the parameter estimates for different groups. This dissertation also discusses some general techniques to handle measurement error modelling with different IRT models, including the three-parameter normal ogive model and polytomous response models. Several methods of estimating DIF are studied as well. Large sample properties are established to justify the proposed methods. Extensive simulation studies show that the resulting methods perform well in terms of Type-I error rate control, accuracy in estimating DIF and power against both unidirectional and crossing DIF.

Curvilinearity in the Reference Composite and Practical Implications for Measurement

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

Robust Measurement Via A Fused Latent and Graphical Item Response Theory Model

Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

Understanding Ability and Reliability Differences Measured with Count Items: The Distributional Regression Test Model and the Count Latent Regression Model

On maximizing item information and matching difficulty with ability

Effects of the Quantity and Magnitude of Cross-Loading and Model Specification on MIRT Item Parameter Recovery

Alternative Approaches to Addressing Non-Normal Distributions in the Application of IRT Models to Personality Measures

Statistical Detection and Estimation of Differential Item Functioning in Computerized Adaptive Testing

Flexible Bayesian modelling in dichotomous item response theory using mixtures of skewed item curves

Reducing Attenuation Bias in Regression Analyses Involving Rating Scale Data via Psychometric Modeling

Modeling construct change over time amidst potential changes in construct measurement: A longitudinal moderated factor analysis approach

Benefits of the Curious Behavior of Bayesian Hierarchical Item Response Theory Models—An in-Depth Investigation and Bias Correction

Measurement Invariance and Differential Item Functioning in Latent Class Analysis With Stepwise Multiple Indicator Multiple Cause Modeling

Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models.

Bayesian penalty methods for evaluating measurement invariance in moderated nonlinear factor analysis.

Improving measurement-invariance assessments: correcting entrenched testing deficiencies

DIF Statistical Inference Without Knowing Anchoring Items

How does item wording affect participants' responses in Likert scale? Evidence from IRT analysis