Abstract:The ability to rapidly provide examinees with detailed and effective diagnostic information is a critical topic in psychology. Knowing what diagnostic criteria the examinees have met enables the practitioner to seek the solution to help them in a timely manner, and this can be achieved by cognitive diagnostic computerized adaptive testing (CD-CAT). However, the pervasive challenge of replenishing items in the CD-CAT item bank limits its practical application. Online calibration is a means to address item replenishment, but in CD-CAT, most existing online calibration methods that jointly calibrate the Q-matrix and item parameters of the new items are developed only for dichotomous responses and are time-consuming. Notably, previous studies pay no attention to polytomously scored items that are frequently observed in testing, even though they can offer additional evidence for the examinees' diagnosis. To fill this gap, we propose a SCAD-based method (SCAD-EM) to calibrate the Q-matrix and item parameters of the new items with polytomous response data in order to promote the application of CD-CAT in practice. The performance of the SCAD-EM was investigated in two comprehensive simulation studies and compared against the revised single-item estimation method (SIE-BIC). Results indicated that the SCAD-EM produces a higher calibration accuracy for the category-level Q-matrix and is computationally more efficient across all conditions, but it produces a lower calibration accuracy for the item-level Q-matrix. An empirical study further demonstrated the utility of the SCAD-EM and the SIE-BIC methods in calibrating new items with a real dataset. The advantages of the proposed method, its limitations, and possible future research directions are offered at the end.

Exploration of Polytomous-attribute Q -matrix Validation in Cognitive Diagnostic Assessment

Validating a Reading Assessment Within the Cognitive Diagnostic Assessment Framework: Q-Matrix Construction and Model Comparisons for Different Primary Grades

Combining regularization and logistic regression model to validate the Q‐matrix for cognitive diagnosis model

Regularized Bayesian algorithms for Q‐matrix inference based on saturated cognitive diagnosis modelling

Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

Using country-specific Q-matrices for cognitive diagnostic assessments with international large-scale data

Learning Item-Attribute Relationship in Q-Matrix Based Diagnostic Classification Models

Data-Driven Learning of Q-Matrix.

Hypothesis Testing of the Q-matrix

Applying support vector machines to a diagnostic classification model for polytomous attributes in small‐sample contexts

A Two-Step Q-Matrix Estimation Method

The Impact of Q-Matrix Designs on Diagnostic Classification Accuracy in the Presence of Attribute Hierarchies

A Semi-supervised Learning Method for Q-Matrix Specification Under the DINA and DINO Model With Independent Structure

Statistical Analysis Ofq-Matrix Based Diagnostic Classification Models

Constructing and Validating a Q-Matrix for Cognitive Diagnostic Analyses of a Reading Test

An exploratory Q-matrix estimation method based on sparse non-negative matrix factorization

Cognitive diagnostic assessment: A Q-matrix constraint-based neural network method

Non-identifiability, Equivalence Classes, and Attribute-Specific Classification in Q-matrix Based Cognitive Diagnosis Models

An Application of the Support Vector Machine for Attribute-By-Attribute Classification in Cognitive Diagnosis

Variational Bayesian Inference for a Polytomous-Attribute Saturated Diagnostic Classification Model with Parallel Computing

Mutual Information Item Selection Method in Cognitive Diagnostic Computerized Adaptive Testing With Short Test Length