Abstract:Applied Psychological Measurement, Ahead of Print. Published information is limited regarding the security of testing programs, and even less on the relative security of different testing modalities: in-person at test centers (TC) versus remote online proctored (OP) testing. This article begins by examining indicators of test security violations across a wide range of programs in professional, admissions, and IT fields. We look at high levels of response overlap as a potential indicator of collusion to cheat on the exam and compare rates by modality and between test center types. Next, we scrutinize indicators of potential test security violations for a single large testing program over the course of 14 months, during which the program went from exclusively in-person TC testing to a mix of OP and TC testing. Test security indicators include high response overlap, large numbers of fast correct responses, large numbers of slow correct responses, large test-retest score gains, unusually fast response times for passing candidates, and measures of differential person functioning. These indicators are examined and compared prior to and after the introduction of OP testing. In addition, test-retest modality is examined for candidates who fail and retest subsequent to the introduction of OP testing, with special attention paid to test takers who change modality between the initial attempt and the retest. These data allow us to understand whether indications of content exposure increase with the introduction of OP testing, and whether testing modalities affect potential score increase in a similar way.

A Mark-Recapture Approach to Estimating Item Pool Compromise

Detecting Compromised Items With Response Times Using a Bayesian Change-Point Approach

Using Item Scores and Distractors to Detect Item Compromise and Preknowledge

Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing

Detecting Item Preknowledge Using a Predictive Checking Method

Detecting Aberrant Behavior and Item Preknowledge: A Comparison of Mixture Modeling Method and Residual Method

Monte Carlo detection of examinees with item preknowledge

Are Exam Questions Known in Advance? Using Local Dependence to Detect Cheating

Item Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

Estimating Test-Retest Reliability in the Presence of Self-Selection Bias and Learning/Practice Effects

A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing

A Model-Based Approach to the Disentanglement and Differential Treatment of Engaged and Disengaged Item Omissions

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

A Mixture Response Model for Identifying Item Preknowledge

A Robust Computerized Adaptive Testing Approach in Educational Question Retrieval

Test Security and the Pandemic: Comparison of Test Center and Online Proctor Delivery Modalities

Using Deterministic, Gated Item Response Theory Model to detect test cheating due to item compromise

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts

The Effect of Missing Item Data on the Relative Predictive Accuracy of Correctional Risk Assessment Tools

Location-Matching Adaptive Testing for Polytomous Technology-Enhanced Items

A Mixture Response Time Process Model for Aberrant Behaviors and Item Nonresponses