Disentangling Heterogeneous Knowledge Concept Embedding for Cognitive Diagnosis on Untested Knowledge

Miao Zhang,Ziming Wang,Runtian Xing,Kui Xiao,Zhifei Li,Yan Zhang,Chang Tang

2024-10-18

Abstract:Cognitive diagnosis is a fundamental and critical task in learning assessment, which aims to infer students' proficiency on knowledge concepts from their response logs. Current works assume each knowledge concept will certainly be tested and covered by multiple exercises. However, whether online or offline courses, it's hardly feasible to completely cover all knowledge concepts in several exercises. Restricted tests lead to undiscovered knowledge deficits, especially untested knowledge concepts(UKCs). In this paper, we propose a novel framework for Cognitive Diagnosis called Disentangling Heterogeneous Knowledge Cognitive Diagnosis(DisKCD) on untested knowledge. Specifically, we leverage course grades, exercise questions, and learning resources to learn the potential representations of students, exercises, and knowledge concepts. In particular, knowledge concepts are disentangled into tested and untested based on the limiting actual exercises. We construct a heterogeneous relation graph network via students, exercises, tested knowledge concepts(TKCs), and UKCs. Then, through a hierarchical heterogeneous message-passing mechanism, the fine-grained relations are incorporated into the embeddings of the entities. Finally, the embeddings will be applied to multiple existing cognitive diagnosis models to infer students' proficiency on UKCs. Experimental results on real-world datasets show that the proposed model can effectively improve the performance of the task of diagnosing students' proficiency on UKCs. Our code is available at <a class="link-external link-https" href="https://github.com/Hubuers/DisKCD" rel="external noopener nofollow">this https URL</a>.

Artificial Intelligence,Computers and Society,Machine Learning

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem of evaluating untested knowledge concepts (UKCs, Untested Knowledge Concepts) in cognitive diagnosis. Specifically, existing cognitive diagnosis methods usually assume that each knowledge concept will be tested through multiple practice questions. However, in actual online or offline courses, it is difficult to fully cover all knowledge concepts. This results in many knowledge concepts not being fully tested, thus affecting the accurate assessment of students' mastery of these untested knowledge concepts.

#### Main problems:

1. **Evaluation of untested knowledge concepts**:
- In actual examinations, due to time and examination scope limitations, many knowledge concepts cannot be fully covered, forming untested knowledge blind spots.
- These untested knowledge concepts may have a profound impact on students' overall performance and knowledge structure, but traditional assessment methods often overlook this.

2. **Comprehensive understanding of students' learning progress**:
- Relying solely on tested knowledge concepts (TKCs, Tested Knowledge Concepts) for assessment may lead teachers to be unable to fully understand students' learning progress and miss opportunities for timely intervention and personalized guidance.
- Therefore, effectively diagnosing students' mastery of untested knowledge concepts is crucial. This not only helps identify knowledge gaps but also provides key data for designing personalized learning paths.

#### Solutions:

To address the above challenges, the paper proposes a new cognitive diagnosis framework - Disentangling Heterogeneous Knowledge Cognitive Diagnosis (DisKCD), which can effectively evaluate students' mastery of untested knowledge concepts. Specific methods include:

- **Distinguishing between tested and untested knowledge concepts**: Divide knowledge concepts into two categories: tested (TKCs) and untested (UKCs).
- **Constructing a heterogeneous relationship graph network**: Construct a heterogeneous relationship graph network through students, practice questions, tested knowledge concepts, and untested knowledge concepts to capture the complex relationships between them.
- **Hierarchical heterogeneous message - passing mechanism**: Through this mechanism, fine - grained relationship information is integrated into entity embedding representations to enhance the expressive power of the embedding representations.
- **Application to multiple cognitive diagnosis models**: Apply the enhanced embedding representations to existing cognitive diagnosis models to predict students' performance on untested practice questions, thereby evaluating their mastery of untested knowledge concepts.

Through this method, DisKCD can more comprehensively evaluate students' learning states, help educators better understand students' learning progress, and provide more targeted teaching support.