Abstract:Item preknowledge describes a scenario where candidates may have access to some of the test items prior to the test administration. This involves sharing test materials and/or answers and it is difficult to identify the individuals with item preknowledge or the shared materials of the test. Nevertheless, it is essential to investigate the ‘item preknowledge’ problem because it can significantly affect the validity of the test results. It is believed that traditional linear tests are more robust to this type of aberrant response behavior than adaptive tests. In this context, the aim of this study is to examine the effect of item preknowledge on computer adaptive tests and identify the conditions under which adaptive tests are most resistant to the item preknowledge. For this purpose, a Monte Carlo simulation study was performed and 28 different conditions were examined. The results of the study indicated that the EAP estimation method provided better measurement precision than ML over all conditions. When 2PL and 3PL IRT models were compared, it was observed that 2PL had higher precision at most of the conditions. However, when the aberrancy ratio increased and reached 20% for both individuals and items, 3PL outperformed the 2PL model and gave the best results with the EAP combination. The results were discussed in line with the literature on item preknowledge and CAT and implications for practitioners and further research were provided.

Two New Models for Item Preknowledge

A Mixture Response Model for Identifying Item Preknowledge

Generating Models for Item Preknowledge

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

Detecting Item Preknowledge Using Revisits With Speed and Accuracy

Detection of Item Preknowledge Using Response Times

Robustness of Computer Adaptive Tests to the Presence of Item Preknowledge: A Simulation Study

Concurrent Use of Response Time and Response Accuracy for Detecting Examinees with Item Preknowledge

Detection of Item Preknowledge Using Likelihood Ratio Test and Score Test

A Machine-Learning-Based Approach for Detecting Item Preknowledge in Computerized Adaptive Testing

Detecting Aberrant Behavior and Item Preknowledge: A Comparison of Mixture Modeling Method and Residual Method

Detecting Item Preknowledge Using a Predictive Checking Method

Comparing the Performance of Eight Item Preknowledge Detection Statistics

The Effect of Item Preknowledge on Response Time: Analysis of an Experimental Dataset Using the Multiple-Group Lognormal Response Time Model with a Gating Mechanism

Using Item Scores and Distractors to Detect Item Compromise and Preknowledge

A Deterministic Gated Lognormal Response Time Model to Identify Examinees with Item Preknowledge

Monte Carlo detection of examinees with item preknowledge

Probabilistic Models for Computerized Adaptive Testing

Which Statistic Should Be Used to Detect Item Preknowledge When the Set of Compromised Items Is Known?

Are Exam Questions Known in Advance? Using Local Dependence to Detect Cheating