Abstract:In low-stakes assessments, some students may not reach the end of the test and leave some items unanswered due to various reasons (e.g., lack of test-taking motivation, poor time management, and test speededness). Not-reached items are often treated as incorrect or not-administered in the scoring process. However, when the proportion of not-reached items is high, these traditional approaches may yield biased scores and thereby threatening the validity of test results. In this study, we propose a polytomous scoring approach for handling not-reached items and compare its performance with those of the traditional scoring approaches. Real data from a low-stakes math assessment administered to second and third graders were used. The assessment consisted of 40 short-answer items focusing on addition and subtraction. The students were instructed to answer as many items as possible within 5 minutes. Using the traditional scoring approaches, students’ responses for not-reached items were treated as either not-administered or incorrect in the scoring process. With the proposed scoring approach, students’ nonmissing responses were scored polytomously based on how accurately and rapidly they responded to the items to reduce the impact of not-reached items on ability estimation. The traditional and polytomous scoring approaches were compared based on several evaluation criteria, such as model fit indices, test information function, and bias. The results indicated that the polytomous scoring approaches outperformed the traditional approaches. The complete case simulation corroborated our empirical findings that the scoring approach in which nonmissing items were scored polytomously and not-reached items were considered not-administered performed the best. Implications of the polytomous scoring approach for low-stakes assessments were discussed.

Modeling Not-Reached Items in Timed Tests: A Response Time Censoring Approach.

Contributions to the statistical analysis of item response time in educational testing

A Mixture Response Time Process Model for Aberrant Behaviors and Item Nonresponses

A Model-Based Approach to the Disentanglement and Differential Treatment of Engaged and Disengaged Item Omissions

Modeling Rapid Guessing Behaviors in Computer-Based Testlet Items

A Mixture Cure-Rate Model for Responses and Response Times in Time-Limit Tests

Generalized Residual-Based Diagnostic Testing for Duration Models with Censoring

Utilizing Response Time for Item Selection in On‐the‐Fly Multistage Adaptive Testing for PISA Assessment

Item Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

Detecting Compromised Items With Response Times Using a Bayesian Change-Point Approach

Testing for Aberrant Behavior in Response Time Modeling

Measuring Motivation to Take Low-Stakes Large-Scale Test: New Model Based on Analyses of “Participant-Own-defined” Missingness

A Polytomous Scoring Approach to Handle Not-Reached Items in Low-Stakes Assessments

A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing

Detecting Aberrant Behavior and Item Preknowledge: A Comparison of Mixture Modeling Method and Residual Method

A Simulation-based Study of the Effects of Incomplete Data on Lifetime Distribution Estimation due to Censorship

Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing

Semiparametric Estimation Method for Accelerated Failure Time Model with Dependent Censoring.

Does Timed Testing Affect the Interpretation of Efficiency Scores?—A GLMM Analysis of Reading Components

Efficiency of computerized adaptive testing with a cognitively designed item bank

Regression Analysis of Right-Censored Failure Time Data with Missing Censoring Indicators