Abstract:In high-stakes, large-scale, standardized tests with certain time limits, examinees are likely to engage in either one of the three types of behavior (e.g., van der Linden & Guo, 2008; Wang & Xu, 2015): solution behavior, rapid guessing behavior, and cheating behavior. Oftentimes examinees do not always solve all items due to various reasons such as time limit or test-taking strategy. Item nonresponses may happen due to intentionally omitting some items (omitted responses) or due to lack of time (not-reached responses). Both types are related to latent abilities and hence the missingness is nonignorable. In this article, we proposed an innovative mixture response time process model (1) to detect two most common aberrant behaviors: rapid guessing behavior and cheating behavior, and (2) to account for two types of item nonresponses: not-reached items and omitted items. The new model combines the two-stage approach of Wang et al. (2018) with Lu and Wang (2020) model. It also contains two steps: (1) a mixture response time process model is first fitted to the responses and response times data to distinguish normal and aberrant behaviors and to account for the missing data mechanism; and (2) a Bayesian residual index is used to further distinguish rapid guessing and cheating behaviors. Simulation results show that the two-stage method yields accurate item and person parameter estimates, as well as high detection of aberrant behaviors. A real data analysis was conducted to illustrate the potential application of the proposed method.

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

A Mixture Response Model for Identifying Item Preknowledge

Detecting Aberrant Behavior and Item Preknowledge: A Comparison of Mixture Modeling Method and Residual Method

A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing

Multimodal Data Fusion to Detect Preknowledge Test-Taking Behavior Using Machine Learning

Concurrent Use of Response Time and Response Accuracy for Detecting Examinees with Item Preknowledge

Using Deterministic, Gated Item Response Theory Model to detect test cheating due to item compromise

A Mixture Response Time Process Model for Aberrant Behaviors and Item Nonresponses

Two New Models for Item Preknowledge

A Machine-Learning-Based Approach for Detecting Item Preknowledge in Computerized Adaptive Testing

Are Exam Questions Known in Advance? Using Local Dependence to Detect Cheating

Detection of Item Preknowledge Using Response Times

Cheating Detection Method Based On Improved Cognitive Diagnosis Model

Detecting Item Preknowledge Using Revisits With Speed and Accuracy

Detection of Item Preknowledge Using Likelihood Ratio Test and Score Test

Detecting Item Preknowledge Using a Predictive Checking Method

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

Detecting Examinees With Item Preknowledge in Large-Scale Testing Using Extreme Gradient Boosting (XGBoost)

Exploration of the Stacking Ensemble Machine Learning Algorithm for Cheating Detection in Large-Scale Assessment

Detection of cheating by decimation algorithm