Abstract:The computerized adaptive form of cognitive diagnostic testing, CD-CAT, has gained increasing attention in the domain of personalized measurements for its ability to categorize individual mastery status of fine-grained attributes more accurately and efficiently through administering items tailored to one's ability progressively. How to select the next item based on previous response(s) is crucial for the success of CD-CAT. Previous item selection strategies for CD-CAT have often followed a greedy or semi-greedy approach, which makes it difficult to strike a balance between diagnostic performance and item bank utilization. To address this issue, this study takes a graph perspective and transforms the item selection problem in CD-CAT into a path-searching problem, in which paths refer to possible test construction and nodes refer to individual items. A heuristic function is defined to predict the prospect of a path, indicating how well the corresponding test can diagnose the current examinee. Two search mechanisms with different biases towards item exposure control are proposed to approximate the optimal path with the best prospect. The first unused item on the resulting path is selected as the next item. The above components compose a novel CD-CAT item selection framework based on heuristic search. Simulation studies are conducted under a variety of conditions regarding bank designs, bank-quality conditions, and testing scenarios. The results are compared with different types of classic item selection strategies in CD-CAT, showing that the proposed framework can enhance bank utilization at a smaller cost of diagnostic performance.

Monitoring Item Performance With CUSUM Statistics in Continuous Testing

Item Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

Detecting uniform differential item functioning for continuous response computerized adaptive testing

Efficiency of computerized adaptive testing with a cognitively designed item bank

Detecting Item Preknowledge Using a Predictive Checking Method

Detecting Item Preknowledge Using Revisits With Speed and Accuracy

CUSUM Procedures for Health Care Surveillance.

Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing

Comparing the Performance of Eight Item Preknowledge Detection Statistics

Detecting Aberrant Behavior and Item Preknowledge: A Comparison of Mixture Modeling Method and Residual Method

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

SCR-CUSUM: an Illness-Death Semi-Markov Model-Based Risk-Adjusted CUSUM for Semi-Competing Risk Data Monitoring

The Impact of Varying Patient Populations on the In-Control Performance of the Risk-Adjusted Cusum Chart

CGR-CUSUM: a continuous time generalized rapid response cumulative sum chart

A Comparison of Detection Probability of CUSUM Procedures under Increasing Population Sizes

Weighted CUSUM Control Charts for Monitoring Poisson Processes with Varying Sample Sizes

Parametric and nonparametric versions of adaptive CUSUM charts for monitoring the location of individual measurements

Detection of Item Preknowledge Using Likelihood Ratio Test and Score Test

A binomial CUSUM chart for detecting large shifts in fraction nonconforming

A two‐step item bank calibration strategy based on 1‐bit matrix completion for small‐scale computerized adaptive testing

Sequential change point tests based on U-statistics