Novel Use of Self-organizing Map for Q-matrix Calibration in Cognitive Diagnosis Assessment

Xi-tian Chen,Zhengjia Dai,Ying Lin

DOI: https://doi.org/10.1109/ijcnn48605.2020.9207587

2020-01-01

Abstract:Cognitive diagnostic assessment (CDA) is a newly emerged tool for educational and psychological measurement. Unlike traditional assessment, CDA can measure a participant's detailed mastering pattern of finer-grained skills and knowledge (a.k.a. attributes). An important component in CDA is the Q-matrix, which associates each item with particular attribute(s) and thus forms the basis for performing diagnosis. However, in real-world applications, the Q-matrix is often defined by hand, which induces high risks of subjective errors. The purpose of the current investigation is to describe how the self-organizing map (SOM), a typical kind of unsupervised artificial neural network, can be used to validate and calibrate the Q-matrix. To do so, we propose to convert the Q-matrix calibration problem into a clustering-labeling process: first cluster the participants' responses to identify groups of test items and then label the item(s) with unknown or obscure attribute(s) based on group similarity. Following this idea, a one-phase method and a two-phase method are proposed. The one-phase method employs a single SOM to implement Q-matrix calibration at a fast speed. The two-phase method, which adopts a double-SOM design, is a bit more time-consuming but can provide a diagnosis of participants as a by-product. Experimental results on simulated data generated under various conditions confirm the effectiveness of the proposed method. As a real-data illustration, the proposed methods are also applied to analyze large-scale educational reading assessment data and achieve promising results.