Item Selection Using the Multiple-Strategy RRUM Based on CD-CAT
Buyun DAI,Minqiang ZHANG,Can JIAO,Guangming LI,Huawei ZHU,Wenyi ZHANG
DOI: https://doi.org/10.3724/sp.j.1041.2015.01511
2015-01-01
Abstract:Cognitive diagnostic computerized adaptive testing (CD-CAT) is an application of cognitive diagnostic testing (CDT) on the computerized adaptive testing (CAT) platform. CD-CAT can provide information on the specific content areas in which a participant needs the most specific help, and it is typically engineered to tailor the test to each participant's trait level and thus has high efficiency. The search for the best item selection method is currently one of the most pressing issues in the field of CD-CAT research. Besides, studies on CD-CAT have to address the issue on the number of testing strategies to employ: either all participants use the same strategy, or different participants use different strategies. In practice, for some tasks, all participants use the same strategy; whereas for other tasks, different participants use different strategies to solve the problem. Because cognitive strategy aptitudes (or cognitive structures) vary among participants, each strategy requires different attributes. For the latter type of tasks, a participant's strategy must be identified before his/her knowledge state (KS, i.e., attribute mastery pattern) can be diagnosed. Strategy diagnosis procedures have not been explored in studies on CD-CAT. Based on the reduced reparameterized unified model (RRUM) and independence attributes, under different lengths of fixed-length CD-CAT and with various numbers of knowledge attributes, using the maximum aposteriori (MAP), the present study explored the use of the multiple-strategy RRUM (MS-RRUM) and the item selection method in the context of multiple strategies. The MS-RRUM extended the traditional single-strategy RRUM by constructingMdifferent Q-matrices. In the MS-RRUM, each strategy for each item involved a set of parameters, including one baseline parameter and several penalty parameters. The MAP method was then generalized to the multiple-strategy context. An item selection method that fitted the multiple-strategy context, named as the multiple-strategy SHE (MSSHE), was proposed here. When a participant responded to an item, the system estimated the participant's strategy according to his/her item response vector and then estimated the participant's KS using the selected strategy. The next item was then selected via the MSSHE method.Using the MS-RRUM along with varying numbers of attributes and test lengths, the strategy and KS recoveries of the MSSHE and the random method were systematically compared. Results showed that the MSSHE method exhibited excellent strategy and KS estimations, and it was superior to the random method. Besides, when the test was short, the advantage of the MSSHE was more obvious in both strategy estimation and KS estimation; when the number of attributes was relative large, the advantage of the MSSHE was more obvious in KS estimation. Therefore, (1) the MS-RRUM is feasible and can be used in the CD-CAT context; (2) the MAP method, when extended to the multiple-strategy context, can successfully estimate participants' strategies and KSs; (3) when CD-CAT is applied to the MS-RRUM, the proposed MSSHE method is feasible and efficient in item selection. This study is the first to apply CD-CAT in a multiple-strategy context, providing more sufficient diagnostic information. Furthermore, the proposed method of MAP in the multiple-strategy context makes parameter estimation easier and is time-saving, and the proposed MSSHE method makes item selection efficient and saves item bank. Overall, this study serves to facilitate the implantation of multiple strategies in CD-CAT. Further studies are suggested to examine the performance of these approaches under other conditions (i.e., using other cognitive diagnostic models, attribute hierarchical structures and variable-length CD-CAT settings).