Abstract:In high-stakes, large-scale, standardized tests with certain time limits, examinees are likely to engage in either one of the three types of behavior (e.g., van der Linden & Guo, 2008; Wang & Xu, 2015): solution behavior, rapid guessing behavior, and cheating behavior. Oftentimes examinees do not always solve all items due to various reasons such as time limit or test-taking strategy. Item nonresponses may happen due to intentionally omitting some items (omitted responses) or due to lack of time (not-reached responses). Both types are related to latent abilities and hence the missingness is nonignorable. In this article, we proposed an innovative mixture response time process model (1) to detect two most common aberrant behaviors: rapid guessing behavior and cheating behavior, and (2) to account for two types of item nonresponses: not-reached items and omitted items. The new model combines the two-stage approach of Wang et al. (2018) with Lu and Wang (2020) model. It also contains two steps: (1) a mixture response time process model is first fitted to the responses and response times data to distinguish normal and aberrant behaviors and to account for the missing data mechanism; and (2) a Bayesian residual index is used to further distinguish rapid guessing and cheating behaviors. Simulation results show that the two-stage method yields accurate item and person parameter estimates, as well as high detection of aberrant behaviors. A real data analysis was conducted to illustrate the potential application of the proposed method.

Detecting Compromised Items With Response Times Using a Bayesian Change-Point Approach

Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing

Item Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

Detecting Aberrant Behavior and Item Preknowledge: A Comparison of Mixture Modeling Method and Residual Method

Detection of Item Preknowledge Using Response Times

Using Item Scores and Distractors to Detect Item Compromise and Preknowledge

A Mark-Recapture Approach to Estimating Item Pool Compromise

Detecting Item Preknowledge Using a Predictive Checking Method

Improvement and application of back random response detection: Based on cumulative sum and change point analysis

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

A Mixture Response Time Process Model for Aberrant Behaviors and Item Nonresponses

A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

Concurrent Use of Response Time and Response Accuracy for Detecting Examinees with Item Preknowledge

A Machine-Learning-Based Approach for Detecting Item Preknowledge in Computerized Adaptive Testing

Detecting uniform differential item functioning for continuous response computerized adaptive testing

A Mixture Response Model for Identifying Item Preknowledge

Measuring the robustness of predictive probability for early stopping in two-group comparisons

Detection of Item Preknowledge Using Likelihood Ratio Test and Score Test

Which Statistic Should Be Used to Detect Item Preknowledge When the Set of Compromised Items Is Known?

Modeling Not-Reached Items in Timed Tests: A Response Time Censoring Approach.