Abstract:Knowledge Tracing (KT) is a critical task in online learning for modeling student knowledge over time. Despite the success of deep learning-based KT models, which rely on sequences of numbers as data, most existing approaches fail to leverage the rich semantic information in the text of questions and concepts. This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that integrates pre-trained language models (PLMs) with KT methods. By leveraging the power of language models to capture semantic representations, LKT effectively incorporates textual information and significantly outperforms previous KT models on large benchmark datasets. Moreover, we demonstrate that LKT can effectively address the cold-start problem in KT by leveraging the semantic knowledge captured by PLMs. Interpretability of LKT is enhanced compared to traditional KT models due to its use of text-rich data. We conducted the local interpretable model-agnostic explanation technique and analysis of attention scores to interpret the model performance further. Our work highlights the potential of integrating PLMs with KT and paves the way for future research in KT domain.

What problem does this paper attempt to address?

The paper aims to address several key issues in the field of Knowledge Tracing (KT): 1. **Utilizing Text Information**: Existing deep learning KT models primarily rely on numerical sequences to represent Knowledge Concepts (KCs) and questions, failing to fully leverage the rich semantic information in the text of questions and concepts. 2. **Cold Start Problem**: Many KT models need to be trained from scratch when faced with new domains or new data, as they struggle to take advantage of pre-trained models, leading to the cold start problem. 3. **Interpretability Issue**: The interpretability of current KT models is limited by their reliance on numerical sequences, which lack intuitive semantic meaning. Compared to Natural Language Processing (NLP) models, they fall short in applying Explainable Artificial Intelligence (XAI) techniques. To address the above challenges, the paper proposes a new framework—Language model-based Knowledge Tracing (LKT). This framework effectively incorporates the textual information of questions and concepts into the model by integrating Pre-trained Language Models (PLMs) with KT methods. LKT leverages the powerful capability of PLMs to capture semantic representations, significantly improving the performance of KT tasks, better addressing the cold start problem, and enhancing model interpretability. Specifically, the key contributions of LKT include: - Proposing a new LKT framework that combines PLMs with KT tasks, enabling accurate prediction of student performance on new questions and concepts with limited data. - Providing insights into which knowledge points and question parts affect student performance through in-depth analysis of model performance, which is valuable for learning scientists and educational researchers. - Experimental results show that LKT significantly outperforms existing KT models on large benchmark datasets, especially excelling in solving the cold start problem. - The study also explores the impact of different sizes of language models on performance, finding that larger models can achieve higher AUC scores with appropriate training strategies. - LKT enhances model interpretability through attention mechanisms and Local Interpretable Model-agnostic Explanations (LIME) techniques, aiding in understanding the important basis of model prediction results.

Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task

From Prediction to Application: Language Model-based Code Knowledge Tracing with Domain Adaptive Pre-Training and Automatic Feedback System with Pedagogical Prompting for Comprehensive Programming Education

A Deeper Knowledge Tracking Model Integrating Cognitive Theory and Learning Behavior

Knowledge Tracing As Language Processing: A Large-Scale Autoregressive Paradigm

CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer

A Systematic Review of Knowledge Tracing and Large Language Models in Education: Opportunities, Issues, and Future Research

Towards Interpretable Deep Learning Models for Knowledge Tracing

Interpreting Deep Learning Models for Knowledge Tracing

Programming Knowledge Tracing: A Comprehensive Dataset and A New Model

Difficulty-Focused Contrastive Learning for Knowledge Tracing with a Large Language Model-Based Difficulty Prediction

Mitigating Cold-Start Problems in Knowledge Tracing with Large Language Models: an Attribute-aware Approach

Beyond Right and Wrong: Mitigating Cold Start in Knowledge Tracing Using Large Language Model and Option Weight

Knowledge ontology enhanced model for explainable knowledge tracing

A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing Models

Deep Attentive Model for Knowledge Tracing.

Integrating LSTM and BERT for Long-Sequence Data Analysis in Intelligent Tutoring Systems

Interpretable Knowledge Tracing: Simple and Efficient Student Modeling with Causal Relations

Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations

Knowledge-Associated Embedding for Memory-Aware Knowledge Tracing

Recent Advances on Deep Learning Based Knowledge Tracing

DCKT: A Novel Dual-Centric Learning Model for Knowledge Tracing