Abstract:The ability to rapidly provide examinees with detailed and effective diagnostic information is a critical topic in psychology. Knowing what diagnostic criteria the examinees have met enables the practitioner to seek the solution to help them in a timely manner, and this can be achieved by cognitive diagnostic computerized adaptive testing (CD-CAT). However, the pervasive challenge of replenishing items in the CD-CAT item bank limits its practical application. Online calibration is a means to address item replenishment, but in CD-CAT, most existing online calibration methods that jointly calibrate the Q-matrix and item parameters of the new items are developed only for dichotomous responses and are time-consuming. Notably, previous studies pay no attention to polytomously scored items that are frequently observed in testing, even though they can offer additional evidence for the examinees' diagnosis. To fill this gap, we propose a SCAD-based method (SCAD-EM) to calibrate the Q-matrix and item parameters of the new items with polytomous response data in order to promote the application of CD-CAT in practice. The performance of the SCAD-EM was investigated in two comprehensive simulation studies and compared against the revised single-item estimation method (SIE-BIC). Results indicated that the SCAD-EM produces a higher calibration accuracy for the category-level Q-matrix and is computationally more efficient across all conditions, but it produces a lower calibration accuracy for the item-level Q-matrix. An empirical study further demonstrated the utility of the SCAD-EM and the SIE-BIC methods in calibrating new items with a real dataset. The advantages of the proposed method, its limitations, and possible future research directions are offered at the end.

Exploration of Item Selection in Dual-Purpose Cognitive Diagnostic Computerized Adaptive Testing: Based on the RRUM.

Mutual Information Item Selection Method in Cognitive Diagnostic Computerized Adaptive Testing With Short Test Length

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

Nonparametric CD‐CAT for multiple‐choice items: Item selection method and Q‐optimality

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

An adaptive testing item selection strategy via a deep reinforcement learning approach

A Robust Computerized Adaptive Testing Approach in Educational Question Retrieval

Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

Efficiency of computerized adaptive testing with a cognitively designed item bank

Application of Bi-factor MIRT and Higher-order CDM Models to an In-house EFL Listening Test for Diagnostic Purposes

Detecting uniform differential item functioning for continuous response computerized adaptive testing

Item Response Ranking for Cognitive Diagnosis

Iterative Item Selection of Neighborhood Clusters: A Nonparametric and Non-IRT Method for Generating Miniature Computer Adaptive Questionnaires

Multidimensional Computerized Adaptive Testing for Classifying Examinees With Within-Dimensionality

Design and Application of a Response-Driven Adaptive Test System

Modified Cognitive Diagnostic Index and Modified Attribute-Level Discrimination Index for Test Construction

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models

An Optimal Choice of Cognitive Diagnostic Model for Second Language Listening Comprehension Test

Computerized Adaptive Testing for Sleep Disorders: Development of An Item Bank and Validation in A Simulated Study

A-Stratified Multistage Computerized Adaptive Testing

The selection of cognitive diagnostic models for a reading comprehension test