Abstract:Differential item functioning (DIF) is an important issue in large scale standardized testing. DIF refers to the unexpected difference in item performances among groups of equally proficient examinees, usually classified by ethnicity or gender. Its presence could seriously affect the validity of inferences drawn from a test. Various statistical methods have been proposed to detect and estimate DIF. This dissertation addresses DIF analysis in the context of computerized adaptive testing (CAT), whose item selection algorithm adapts to the ability level of each individual examinee. In a CAT, a DIF item may be more consequential and more detrimental be cause fewer items are administered in a CAT than in a traditional paper-and-pencil test and because the remaining sequence of items presented to examinees depends in part on their responses to the DIF item. Consequently, an efficient, stable and flexible method to detect and estimate CAT DIF becomes necessary and increasingly important. We propose simultaneous implementations of online calibration and DIF testing. The idea is to perform online calibration of an item of interest separately in the focal and reference groups. Under any specific parametric IRT model, we can use the (online) estimated latent traits as covariates and fit a nonlinear regression model to each of the two groups. Because of the use of the estimated, not the true t, the regression fit has to adjust for the covariate “measurement errors”. It turns out that this situation fits nicely into the framework of nonlinear error-in-variable modelling, which has been extensively studied in statistical literature. We develop two bias-correction methods using asymptotic expansion and conditional score theory. After correcting the bias caused by measurement error, one can perform a significance test to detect DIF with the parameter estimates for different groups. This dissertation also discusses some general techniques to handle measurement error modelling with different IRT models, including the three-parameter normal ogive model and polytomous response models. Several methods of estimating DIF are studied as well. Large sample properties are established to justify the proposed methods. Extensive simulation studies show that the resulting methods perform well in terms of Type-I error rate control, accuracy in estimating DIF and power against both unidirectional and crossing DIF.

Effects of Scale Transformation and Test‐Termination Rule on the Precision of Ability Estimation in Computerized Adaptive Testing

A Bounded Ability Estimation for Computerized Adaptive Testing

Standard Error of Ability Estimates and the Classification Accuracy and Consistency of Binary Decisions

A Robust Computerized Adaptive Testing Approach in Educational Question Retrieval

The Impact of Item Model Parameter Variations on Person Parameter Estimation in Computerized Adaptive Testing With Automatically Generated Items

Efficiency of computerized adaptive testing with a cognitively designed item bank

Measurement precision and user experience with adaptive versus non-adaptive psychometric tests

The effects of computer self-efficacy, training satisfaction and test anxiety on attitude and performance in computerized adaptive testing

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

A Rate Function Approach to Computerized Adaptive Testing for Cognitive Diagnosis

Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

Effect of Differential Item Functioning on Computer Adaptive Testing Under Different Conditions

A two‐step item bank calibration strategy based on 1‐bit matrix completion for small‐scale computerized adaptive testing

Ability Assessment Based on CAT in Adaptive Learning System

Developing Computerized Adaptive Testing for a National Health Professionals Exam: An Attempt from Psychometric Simulations

Statistical Detection and Estimation of Differential Item Functioning in Computerized Adaptive Testing

To Weight or Not to Weight? Balancing Influence of Initial Items in Adaptive Testing

Evaluating Robust Scale Transformation Methods with Multiple Outlying Common Items under IRT True Score Equating.

New Robust Scale Transformation Methods in the Presence of Outlying Common Items.

Termination Criteria for Grid Multiclassification Adaptive Testing With Multidimensional Polytomous Items

The Influence of Ability Level and Big Five Personality Traits on Examinees' Test-Taking Behaviour in Computerised Adaptive Testing