Abstract:Educational and Psychological Measurement, Ahead of Print. Rapid-guessing behavior in data can compromise our ability to estimate item and person parameters accurately. Consequently, it is crucial to model data with rapid-guessing patterns in a way that can produce unbiased ability estimates. This study proposes and evaluates three alternative modeling approaches that follow the logic of the effort-moderated item response theory model (EM-IRT) to analyze response data with rapid-guessing responses. One is the two-step EM-IRT model, which utilizes the item parameters estimated by respondents without rapid-guessing behavior and was initially proposed by Rios and Soland without further investigation. The other two models are effort-moderated multidimensional models (EM-MIRT), which we introduce in this study and vary as both between-item and within-item structures. The advantage of the EM-MIRT model is to account for the underlying relationship between rapid-guessing propensity and ability. The three models were compared with the traditional EM-IRT model regarding the accuracy of parameter recovery in various simulated conditions. Results demonstrated that the two-step EM-IRT and between-item EM-MIRT model consistently outperformed the traditional EM-IRT model under various conditions, with the two-step EM-IRT estimation generally delivering the best performance, especially for ability and item difficulty parameters estimation. In addition, different rapid-guessing patterns (i.e., difficulty-based, changing state, and decreasing effort) did not affect the performance of the two-step EM-IRT model. Overall, the findings suggest that the EM-IRT model with the two-step parameter estimation method can be applied in practice for estimating ability in the presence of rapid-guessing responses due to its accuracy and efficiency. The between-item EM-MIRT model can be used as an alternative model when there is no significant mean difference in the ability estimates between examinees who exhibit rapid-guessing behavior and those who do not.

2PL Model: Compare Generalized Linear Mixed Model with Latent Variable Model based IRT framework

Joint Maximum Likelihood Estimation for High-dimensional Exploratory Item Response Analysis

The Rank-2PL IRT Models for Forced-Choice Questionnaires: Maximum Marginal Likelihood Estimation with an EM Algorithm

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

An introduction to mixture item response theory models

Item-Weighted Likelihood Method for Measuring Growth in Longitudinal Study With Tests Composed of Both Dichotomous and Polytomous Items

Alternative Approaches to Addressing Non-Normal Distributions in the Application of IRT Models to Personality Measures

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Joint Maximum Likelihood Estimation for High-Dimensional Exploratory Item Factor Analysis

On maximizing item information and matching difficulty with ability

Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

DIF Detection for Multiple Groups: Comparing Three‐Level GLMMs and Multiple‐Group IRT Models

The Impact of Non-Normality on Extraction of Spurious Latent Classes in Mixture IRT Models

Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate Models for IRT Assessment

Parameter Recovery with Marginal Maximum Likelihood and Markov Chain Monte Carlo Estimation for the Generalized Partial Credit Model

Robust Measurement Via A Fused Latent and Graphical Item Response Theory Model

Effects of the Quantity and Magnitude of Cross-Loading and Model Specification on MIRT Item Parameter Recovery

Enhancing Effort-Moderated Item Response Theory Models by Evaluating a Two-Step Estimation Method and Multidimensional Variations on the Model