Abstract:Educational and Psychological Measurement, Ahead of Print. Rapid-guessing behavior in data can compromise our ability to estimate item and person parameters accurately. Consequently, it is crucial to model data with rapid-guessing patterns in a way that can produce unbiased ability estimates. This study proposes and evaluates three alternative modeling approaches that follow the logic of the effort-moderated item response theory model (EM-IRT) to analyze response data with rapid-guessing responses. One is the two-step EM-IRT model, which utilizes the item parameters estimated by respondents without rapid-guessing behavior and was initially proposed by Rios and Soland without further investigation. The other two models are effort-moderated multidimensional models (EM-MIRT), which we introduce in this study and vary as both between-item and within-item structures. The advantage of the EM-MIRT model is to account for the underlying relationship between rapid-guessing propensity and ability. The three models were compared with the traditional EM-IRT model regarding the accuracy of parameter recovery in various simulated conditions. Results demonstrated that the two-step EM-IRT and between-item EM-MIRT model consistently outperformed the traditional EM-IRT model under various conditions, with the two-step EM-IRT estimation generally delivering the best performance, especially for ability and item difficulty parameters estimation. In addition, different rapid-guessing patterns (i.e., difficulty-based, changing state, and decreasing effort) did not affect the performance of the two-step EM-IRT model. Overall, the findings suggest that the EM-IRT model with the two-step parameter estimation method can be applied in practice for estimating ability in the presence of rapid-guessing responses due to its accuracy and efficiency. The between-item EM-MIRT model can be used as an alternative model when there is no significant mean difference in the ability estimates between examinees who exhibit rapid-guessing behavior and those who do not.

A New Item Response Theory Model for Open-Ended Online Homework with Multiple Allowed Attempts

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

Enhancing Effort-Moderated Item Response Theory Models by Evaluating a Two-Step Estimation Method and Multidimensional Variations on the Model

Item response theory in high-stakes pharmacy assessments

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Using Item Response Theory as a Tool in Educational Measurement

Application of Item Response Theory to Mathematics High School Exams in Portugal

Investigating the Impact of Noneffortful Responses on Individual-Level Scores: Can the Effort-Moderated IRT Model Serve as a Solution?

Modelling the Differences in Social and Emotional Skills with Polytomous Explanatory IRT: The Example of Assertiveness Skill

Scalable Learning of Item Response Theory Models

An empirical study of the effect of granting multiple tries for online homework

Bayesian analysis of dynamic item response models in educational testing

Psychometric Properties of the Internet Self-Efficacy Scale: an Application of Item Response Theory

Implicit assessment of language learning during practice as accurate as explicit testing

Evaluation of student proficiency through a multidimensional finite mixture IRT model

Individual random effects model for differences in trait distribution among respondents

Enhancing Item Response Theory for Cognitive Diagnosis

A Bayesian Nonparametric IRT Model

An Item Response Tree Model for Items with Multiple‐Choice and Constructed‐Response Parts

Modeling Rapid Guessing Behaviors in Computer-Based Testlet Items

A Comparative Study of Item Response Theory Models for Mixed Discrete-Continuous Responses