Explainable Few-shot Knowledge Tracing

Haoxuan Li,Jifan Yu,Yuanxin Ouyang,Zhuang Liu,Wenge Rong,Juanzi Li,Zhang Xiong

2024-05-23

Abstract:Knowledge tracing (KT), aiming to mine students' mastery of knowledge by their exercise records and predict their performance on future test questions, is a critical task in educational assessment. While researchers achieved tremendous success with the rapid development of deep learning techniques, current knowledge tracing tasks fall into the cracks from real-world teaching scenarios. Relying heavily on extensive student data and solely predicting numerical performances differs from the settings where teachers assess students' knowledge state from limited practices and provide explanatory feedback. To fill this gap, we explore a new task formulation: Explainable Few-shot Knowledge Tracing. By leveraging the powerful reasoning and generation abilities of large language models (LLMs), we then propose a cognition-guided framework that can track the student knowledge from a few student records while providing natural language explanations. Experimental results from three widely used datasets show that LLMs can perform comparable or superior to competitive deep knowledge tracing methods. We also discuss potential directions and call for future improvements in relevant topics.

Artificial Intelligence,Computation and Language,Computers and Society

What problem does this paper attempt to address?

The paper aims to address several key issues in the field of Knowledge Tracing (KT):

1. **High Data Demand**: Traditional knowledge tracing methods rely on a large amount of student practice records to train models, whereas in actual teaching scenarios, teachers usually need only a small number of practice records to assess students' knowledge mastery.
2. **Lack of Interpretability**: Existing deep learning models mainly rely on black-box prediction models to predict students' future performance, lacking transparency and interpretability. In contrast, teachers can analyze and explain students' knowledge mastery.
3. **Difficulty in Integrating Multidimensional Information**: Existing models find it challenging to effectively integrate multidimensional information from the learning environment (such as student behavior, question text, etc.), limiting their application scope.

To address these issues, the authors propose "Explainable Few-shot Knowledge Tracing." By leveraging the powerful reasoning and generation capabilities of large language models (LLMs), this new approach can not only predict students' future performance based on a small number of student practice records but also provide natural language explanations. Experimental results show that LLMs perform comparably or even better than existing deep learning methods on three commonly used datasets, with better interpretability and adaptability.