[Inhalation anesthesia in a completely closed circuit. A method of reducing the amount of anesthetic].

P. Toft,A. Jørgensen,M. Djernes

1986-03-17

Abstract:

What problem does this paper attempt to address?

Do We Fully Understand Students' Knowledge States? Identifying and Mitigating Answer Bias in Knowledge Tracing

Chaoran Cui,Hebo Ma,Chen Zhang,Chunyun Zhang,Yumo Yao,Meng Chen,Yuling Ma

DOI: https://doi.org/10.48550/arXiv.2308.07779

IF: 14.4

2023-08-15

Artificial Intelligence

Abstract:Knowledge tracing (KT) aims to monitor students' evolving knowledge states through their learning interactions with concept-related questions, and can be indirectly evaluated by predicting how students will perform on future questions. In this paper, we observe that there is a common phenomenon of answer bias, i.e., a highly unbalanced distribution of correct and incorrect answers for each question. Existing models tend to memorize the answer bias as a shortcut for achieving high prediction performance in KT, thereby failing to fully understand students' knowledge states. To address this issue, we approach the KT task from a causality perspective. A causal graph of KT is first established, from which we identify that the impact of answer bias lies in the direct causal effect of questions on students' responses. A novel COunterfactual REasoning (CORE) framework for KT is further proposed, which separately captures the total causal effect and direct causal effect during training, and mitigates answer bias by subtracting the latter from the former in testing. The CORE framework is applicable to various existing KT models, and we implement it based on the prevailing DKT, DKVMN, and AKT models, respectively. Extensive experiments on three benchmark datasets demonstrate the effectiveness of CORE in making the debiased inference for KT.
Interpretable Knowledge Tracing via Response Influence-based Counterfactual Reasoning

Jiajun Cui,Minghe Yu,Bo Jiang,Aimin Zhou,Jianyong Wang,Wei Zhang

2024-05-31

Abstract:Knowledge tracing (KT) plays a crucial role in computer-aided education and intelligent tutoring systems, aiming to assess students' knowledge proficiency by predicting their future performance on new questions based on their past response records. While existing deep learning knowledge tracing (DLKT) methods have significantly improved prediction accuracy and achieved state-of-the-art results, they often suffer from a lack of interpretability. To address this limitation, current approaches have explored incorporating psychological influences to achieve more explainable predictions, but they tend to overlook the potential influences of historical responses. In fact, understanding how models make predictions based on response influences can enhance the transparency and trustworthiness of the knowledge tracing process, presenting an opportunity for a new paradigm of interpretable KT. However, measuring unobservable response influences is challenging. In this paper, we resort to counterfactual reasoning that intervenes in each response to answer \textit{what if a student had answered a question incorrectly that he/she actually answered correctly, and vice versa}. Based on this, we propose RCKT, a novel response influence-based counterfactual knowledge tracing framework. RCKT generates response influences by comparing prediction outcomes from factual sequences and constructed counterfactual sequences after interventions. Additionally, we introduce maximization and inference techniques to leverage accumulated influences from different past responses, further improving the model's performance and credibility. Extensive experimental results demonstrate that our RCKT method outperforms state-of-the-art knowledge tracing methods on four datasets against six baselines, and provides credible interpretations of response influences.

Computers and Society,Artificial Intelligence,Machine Learning
A Deeper Knowledge Tracking Model Integrating Cognitive Theory and Learning Behavior

Fanglan Ma,Changsheng Zhu,Dukui Liu

DOI: https://doi.org/10.3233/jifs-235723

2024-01-01

Journal of Intelligent & Fuzzy Systems

Abstract:Knowledge tracing (KT), which aims to trace human knowledge learning process by using machines, has widely applied in online learning systems. It dynamically models student’s knowledge states in relation to different learning factors through their learning interactions. Recently, KT has attracted many researches attention due to its good performance to using deep learning. Although most of KT models have shown outstanding results, they have limitations: either ignore the human cognitive law and learning behavior, or lack the ability to go deeper modeling to trace knowledge state. In this paper, we propose a deeper knowledge tracking model integrating cognitive theory and learning behavior (CLDKT). It united the advantages of memory network and recurrent neural network of the existing deep learning KT models for modeling student learning. To better implement CLDKT, we add the residual network (ResNet) to realize the deep modeling of learning behaviors. Extensive experiments on three open benchmark datasets to evaluate our model. Experimental results demonstrate that (I) CLDKT outperforms the state-of-the-art KT models on students’ performance prediction. (II) CLDKT can deeper modeling to trace knowledge state owing to the ResNet import. (III) CLDKT has better interpretability and predictability, which proves the effectiveness of the knowledge tracing model integrating cognitive law and learning behavior.
A Temporal Convolutional Knowledge Tracing Model Integrating Forgetting Factors and Item Response Theory

Zhu Changsheng,Po Shichao,Ma Fanglan

DOI: https://doi.org/10.1109/iccect57938.2023.10140232

2023-01-01

Abstract:As the current mainstream model in the field of knowledge tracking, the deep knowledge tracking model based on RNN has insufficient interpretability, long sequence dependence, gradient disappearance and gradient explosion. Aiming at the above problems, a Temporal Convolutional Knowledge Tracing Model Integrating Forgetting Factors and Item Response Theory(TCKT-FI) was proposed. Firstly, TCN is used to extract the students’ knowledge states. Then the exercise embedding vector is obtained, and the relevant weight vector is obtained through calculation. Next combined with the forgetting factors summarized in pedagogy, the knowledge state is forgotten. Finally, the difficulty of the exercises and the students' ability are obtained by using the embedding vector of the exercises and the mastery of the knowledge points, and the probability of the correct answer to the next question is predicted based on the above factors. Comparing the TCKT-FI model with the DKVMN model, CKT model and SAKT model, the results show that the accuracy rate (ACC) and area under the curve (AUC) of the prediction results are significantly improved, and the best effect is increased by about 14 percentage points, and The calculation speed has been improved by 3-6 times.
Target Hierarchy-Guided Knowledge Tracing: Fine-Grained Knowledge State Modeling

Xinjie Sun,Kai Zhang,Shuanghong Shen,Fei Wang,Yuxiang Guo,Qi Liu

DOI: https://doi.org/10.1016/j.eswa.2024.123898

IF: 8.5

2024-01-01

Expert Systems with Applications

Abstract:knowledge Tracing (KT) focuses on modeling the exercise process of students, assessing their knowledge state changes during the exercise process, and further providing targeted guidance for teaching and learning. Current KT models have made significant progress by utilizing deep learning technologies to model relevant attributes of the questions. However, in current knowledge tracing strategies, we have yet to exhaustively explore how to precisely evaluate the hierarchies of students’ knowledge mastery to better discern their current target hierarchy. To address the limitations of the KT model in accurately identifying and positioning target hierarchy, our research focuses on two key areas: first, we dynamically track students’ target hierarchies by evaluating their feedback on questions at different hierarchies ; second, we assess their actual question-solving abilities within these target hierarchies, thereby optimizing our evaluation of their knowledge status. Based on these insights, we propose the Target Hierarchy-guided Knowledge Tracing (THKT) model. In this model, we first incorporate the identification of question hierarchy into the model representation. Then, to avoid homogenization of the target hierarchy, we dynamically track the appropriate hierarchy for students based on their varying feedback to questions of the same hierarchy, thereby pinpointing their learning target hierarchies. Simultaneously, we develop a KCs applied ability learning module that works in conjunction with the target hierarchies to generate interpretable prediction results. The proposed THKT model has been tested and evaluated using three public, real-world educational datasets. The findings clearly demonstrate that our approach shines in the realm of KT prediction tasks, providing significantly interpretability. For broader research, we plan to provide the source code at: https://github.com/xinjiesun-ustc/THKT to ensure accessibility and promote further innovation in this field.
Counterfactual Monotonic Knowledge Tracing for Assessing Students' Dynamic Mastery of Knowledge Concepts

Moyu Zhang,Xinning Zhu,Chunhong Zhang,Wenchen Qian,Feng Pan,Hui Zhao

DOI: https://doi.org/10.1145/3583780.3614827

2023-08-07

Abstract:As the core of the Knowledge Tracking (KT) task, assessing students' dynamic mastery of knowledge concepts is crucial for both offline teaching and online educational applications. Since students' mastery of knowledge concepts is often unlabeled, existing KT methods rely on the implicit paradigm of historical practice to mastery of knowledge concepts to students' responses to practices to address the challenge of unlabeled concept mastery. However, purely predicting student responses without imposing specific constraints on hidden concept mastery values does not guarantee the accuracy of these intermediate values as concept mastery values. To address this issue, we propose a principled approach called Counterfactual Monotonic Knowledge Tracing (CMKT), which builds on the implicit paradigm described above by using a counterfactual assumption to constrain the evolution of students' mastery of knowledge concepts.

Artificial Intelligence
A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing Models

Hengyuan Zhang,Zitao Liu,Chenming Shang,Dawei Li,Yong Jiang

2024-07-06

Abstract:Knowledge tracing (KT) plays a crucial role in predicting students' future performance by analyzing their historical learning processes. Deep neural networks (DNNs) have shown great potential in solving the KT problem. However, there still exist some important challenges when applying deep learning techniques to model the KT process. The first challenge lies in taking the individual information of the question into modeling. This is crucial because, despite questions sharing the same knowledge component (KC), students' knowledge acquisition on homogeneous questions can vary significantly. The second challenge lies in interpreting the prediction results from existing deep learning-based KT models. In real-world applications, while it may not be necessary to have complete transparency and interpretability of the model parameters, it is crucial to present the model's prediction results in a manner that teachers find interpretable. This makes teachers accept the rationale behind the prediction results and utilize them to design teaching activities and tailored learning strategies for students. However, the inherent black-box nature of deep learning techniques often poses a hurdle for teachers to fully embrace the model's prediction results. To address these challenges, we propose a Question-centric Multi-experts Contrastive Learning framework for KT called Q-MCKT. We have provided all the datasets and code on our website at <a class="link-external link-https" href="https://github.com/rattlesnakey/Q-MCKT" rel="external noopener nofollow">this https URL</a>.

Computers and Society,Artificial Intelligence,Machine Learning
Knowledge ontology enhanced model for explainable knowledge tracing

Yao Wang,Yujia Huo,Changxiao Yang,Xingchen Huang,Dawen Xia,Fujian Feng

DOI: https://doi.org/10.1016/j.jksuci.2024.102065

IF: 9.006

2024-05-19

Journal of King Saud University - Computer and Information Sciences

Abstract:Knowledge Tracing (KT) aims to predict learners' future learning outcomes based on their past learning interactions. Deep Knowledge Tracing (DKT) is a technology developed in recent years that employs deep learning techniques to dynamically track students' learning progress and offer personalized learning support. Yet, existing research often miss the complex interplay between student characteristics and knowledge components, limiting interpretability and adaptability in assessments. Addressing this gap, we introduce the Ontology Perceptible Knowledge Tracing (OPKT) model, a novel approach that reconceptualizes student-knowledge interaction by integrating both student and knowledge ontologies directly into the tracing process. This integration allows for a more nuanced representation of learning interactions, significantly enhancing the predictive precision and interpretability of the model. The OPKT model extracts and utilizes ontology features—deriving student attributes from behavioral data and aligning them with knowledge aspects through sophisticated relationship mapping. This dual-layer integration captures the evolution of knowledge states more accurately, facilitating targeted educational interventions. The results of prediction on two publicly available datasets demonstrate that our proposed OPKT model achieves at least a 2% improvement in AUC, accurately assessing students' learning outcomes and knowledge proficiency. Additionally, we conducted dimensionality reduction experiments and ablation studies to validate the interpretability of the model.

computer science, information systems
Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations

Jiahao Chen,Zitao Liu,Shuyan Huang,Qiongqiong Liu,Weiqi Luo

DOI: https://doi.org/10.48550/arXiv.2302.06885

IF: 5.414

2023-02-14

Machine Learning

Abstract:Knowledge tracing (KT) is a crucial technique to predict students' future performance by observing their historical learning processes. Due to the powerful representation ability of deep neural networks, remarkable progress has been made by using deep learning techniques to solve the KT problem. The majority of existing approaches rely on the \emph{homogeneous question} assumption that questions have equivalent contributions if they share the same set of knowledge components. Unfortunately, this assumption is inaccurate in real-world educational scenarios. Furthermore, it is very challenging to interpret the prediction results from the existing deep learning based KT models. Therefore, in this paper, we present QIKT, a question-centric interpretable KT model to address the above challenges. The proposed QIKT approach explicitly models students' knowledge state variations at a fine-grained level with question-sensitive cognitive representations that are jointly learned from a question-centric knowledge acquisition module and a question-centric problem solving module. Meanwhile, the QIKT utilizes an item response theory based prediction layer to generate interpretable prediction results. The proposed QIKT model is evaluated on three public real-world educational datasets. The results demonstrate that our approach is superior on the KT prediction task, and it outperforms a wide range of deep learning based KT models in terms of prediction accuracy with better model interpretability. To encourage reproducible results, we have provided all the datasets and code at \url{https://pykt.org/}.
Programming Knowledge Tracing: A Comprehensive Dataset and A New Model

Renyu Zhu,Dongxiang Zhang,Chengcheng Han,Ming Gao,Xuesong Lu,Weining Qian,Aoying Zhou

DOI: https://doi.org/10.1109/icdmw58026.2022.00048

2022-01-01

Abstract:In this paper, we study knowledge tracing in the domain of programming education and make two important contributions. First, we harvest and publish so far the most comprehensive dataset, namely BePKT, which covers various online behaviors in an OJ system, including programming text problems, knowledge annotations, user-submitted code and system-logged events. Second, we propose a new model PDKT to exploit the enriched context for accurate student behavior prediction. More specifically, we construct a bipartite graph for programming problem embedding, and design an improved pre-training model PLCodeBERT for code embedding, as well as a double-sequence RNN model with exponential decay attention for effective feature fusion. Experimental results on the new dataset BePKT show that our proposed model establishes state-of-the-art performance in programming knowledge tracing. In addition, we verify that our code embedding strategy based on PLCodeBERT is complementary to existing knowledge tracing models to further enhance their accuracy. As a side product, PLCodeBERT also results in better performance in other programming-related tasks such as code clone detection.
Interpretable Knowledge Tracing: Simple and Efficient Student Modeling with Causal Relations

Sein Minn,Jill-Jênn Vie,Koh Takeuchi,Hisashi Kashima,Feida Zhu

DOI: https://doi.org/10.1609/aaai.v36i11.21560

2022-06-28

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Intelligent Tutoring Systems have become critically important in future learning environments. Knowledge Tracing (KT) is a crucial part of that system. It is about inferring the skill mastery of students and predicting their performance to adjust the curriculum accordingly. Deep Learning based models like Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Network (DKVMN) have shown significant predictive performance compared with traditional models like Bayesian Knowledge Tracing (BKT) and Performance Factors Analysis (PFA). However, it is difficult to extract psychologically meaningful explanations from the tens of thousands of parameters in neural networks, that would relate to cognitive theory. There are several ways to achieve high accuracy in student performance prediction but diagnostic and prognostic reasonings are more critical in learning science. In this work, we present Interpretable Knowledge Tracing (IKT), a simple model that relies on three meaningful features: individual skill mastery, ability profile (learning transfer across skills) and problem difficulty by using data mining techniques. IKT’s prediction of future student performance is made using a Tree Augmented Naive Bayes Classifier (TAN), therefore its predictions are easier to explain than deep learning based student models. IKT also shows better student performance prediction than deep learning based student models without requiring a huge amount of parameters. We conduct ablation studies on each feature to examine their contribution to student performance prediction. Thus, IKT has great potential for providing adaptive and personalized instructions with causal reasoning in real-world educational systems.
Leveraging Pedagogical Theories to Understand Student Learning Process with Graph-based Reasonable Knowledge Tracing

Jiajun Cui,Hong Qian,Bo Jiang,Wei Zhang

2024-06-07

Abstract:Knowledge tracing (KT) is a crucial task in intelligent education, focusing on predicting students' performance on given questions to trace their evolving knowledge. The advancement of deep learning in this field has led to deep-learning knowledge tracing (DLKT) models that prioritize high predictive accuracy. However, many existing DLKT methods overlook the fundamental goal of tracking students' dynamical knowledge mastery. These models do not explicitly model knowledge mastery tracing processes or yield unreasonable results that educators find difficulty to comprehend and apply in real teaching scenarios. In response, our research conducts a preliminary analysis of mainstream KT approaches to highlight and explain such unreasonableness. We introduce GRKT, a graph-based reasonable knowledge tracing method to address these issues. By leveraging graph neural networks, our approach delves into the mutual influences of knowledge concepts, offering a more accurate representation of how the knowledge mastery evolves throughout the learning process. Additionally, we propose a fine-grained and psychological three-stage modeling process as knowledge retrieval, memory strengthening, and knowledge learning/forgetting, to conduct a more reasonable knowledge tracing process. Comprehensive experiments demonstrate that GRKT outperforms eleven baselines across three datasets, not only enhancing predictive accuracy but also generating more reasonable knowledge tracing results. This makes our model a promising advancement for practical implementation in educational settings. The source code is available at <a class="link-external link-https" href="https://github.com/JJCui96/GRKT" rel="external noopener nofollow">this https URL</a>.

Artificial Intelligence,Computers and Society,Machine Learning
Enhancing Deep Knowledge Tracing with Auxiliary Tasks

Zitao Liu,Qiongqiong Liu,Jiahao Chen,Shuyan Huang,Boyu Gao,Weiqi Luo,Jian Weng

DOI: https://doi.org/10.48550/arXiv.2302.07942

2023-02-14

Computers and Society

Abstract:Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interactions with intelligent tutoring systems. Recent studies have applied multiple types of deep neural networks to solve the KT problem. However, there are two important factors in real-world educational data that are not well represented. First, most existing works augment input representations with the co-occurrence matrix of questions and knowledge components\footnote{\label{ft:kc}A KC is a generalization of everyday terms like concept, principle, fact, or skill.} (KCs) but fail to explicitly integrate such intrinsic relations into the final response prediction task. Second, the individualized historical performance of students has not been well captured. In this paper, we proposed \emph{AT-DKT} to improve the prediction performance of the original deep knowledge tracing model with two auxiliary learning tasks, i.e., \emph{question tagging (QT) prediction task} and \emph{individualized prior knowledge (IK) prediction task}. Specifically, the QT task helps learn better question representations by predicting whether questions contain specific KCs. The IK task captures students' global historical performance by progressively predicting student-level prior knowledge that is hidden in students' historical learning interactions. We conduct comprehensive experiments on three real-world educational datasets and compare the proposed approach to both deep sequential KT models and non-sequential models. Experimental results show that \emph{AT-DKT} outperforms all sequential models with more than 0.9\% improvements of AUC for all datasets, and is almost the second best compared to non-sequential models. Furthermore, we conduct both ablation studies and quantitative analysis to show the effectiveness of auxiliary tasks and the superior prediction outcomes of \emph{AT-DKT}.
Beyond Right and Wrong: Mitigating Cold Start in Knowledge Tracing Using Large Language Model and Option Weight

JongWoo Kim,SeongYeub Chu,Bryan Wong,Mun Yi

2024-10-15

Abstract:Knowledge Tracing (KT) is vital in educational data mining, enabling personalized learning by tracking learners' knowledge states and forecasting their academic outcomes. This study introduces the LOKT (Large Language Model Option-weighted Knowledge Tracing) model to address the cold start problem where limited historical data available using large language models (LLMs). While traditional KT models have incorporated option weights, our research extends this by integrating these weights into an LLM-based KT framework. Moving beyond the binary classification of correct and incorrect responses, we emphasize that different types of incorrect answers offer valuable insights into a learner's knowledge state. By converting these responses into text-based ordinal categories, we enable LLMs to assess learner understanding with greater clarity, although our approach focuses on the final knowledge state rather than the progression of learning over time. Using five public datasets, we demonstrate that the LOKT model sustains high predictive accuracy even with limited data, effectively addressing both "learner cold-start" and "system cold-start" scenarios. These findings showcase LOKT's potential to enhance LLM-based learning tools and support early-stage personalization.

Computation and Language,Artificial Intelligence,Computers and Society,Machine Learning
Deep knowledge tracing with learning curves

Hang Su,Xin Liu,Shanghui Yang,Xuesong Lu

DOI: https://doi.org/10.3389/fpsyg.2023.1150329

IF: 3.8

2023-03-31

Frontiers in Psychology

Abstract:Knowledge tracing (KT) models students' mastery level of knowledge concepts based on their responses to the questions in the past and predicts the probability that they correctly answer subsequent questions in the future. Recent KT models are mostly developed with deep neural networks and have demonstrated superior performance over traditional approaches. However, they ignore the explicit modeling of the learning curve theory, which generally says that more practices on the same knowledge concept enhance one's mastery level of the concept. Based on this theory, we propose a Convolution-Augmented Knowledge Tracing (CAKT) model and a Capsule-Enhanced CAKT (CECAKT) model to enable learning curve modeling. In particular, when predicting a student's response to the next question associated with a specific knowledge concept, CAKT uses a module built with three-dimensional convolutional neural networks to learn the student's recent experience on that concept, and CECAKT improves CAKT by replacing the global average pooling layer with capsule networks to prevent information loss. Moreover, the two models employ LSTM networks to learn the overall knowledge state, which is fused with the feature learned by the convolutional/capsule module. As such, the two models can learn the student's overall knowledge state as well as the knowledge state of the concept in the next question. Experimental results on four real-life datasets show that CAKT and CECAKT both achieve better performance compared to existing deep KT models.

psychology, multidisciplinary
Exploiting Multiple Question Factors for Knowledge Tracing

Yan Zhao,Huifang Ma,Wentao Wang,Weiwei Gao,Fanyi Yang,Xiangchun He

DOI: https://doi.org/10.1016/j.eswa.2023.119786

IF: 8.5

2023-03-09

Expert Systems with Applications

Abstract:Knowledge Tracing (KT) aims to predict future students' performance via their responses to a sequence of questions, which serves as a fundamental task for intelligent education. Most of the existing efforts directly predict students' performance depending on their dynamically changing knowledge states. However, the individualization of questions is neglected and difficulty level differ from question to question, which would give some valuable clues to KT. Towards this end, in this paper, we propose a novel Multiple Question Factors for Knowledge Tracing (MQFKT) method, which fully exploits various question factors to generate better prediction. On one hand, calibrated student-concept connection space is established to obtain finer-grained response representations on questions according to the information of responses on questions. On the other hand, individualized difficulty levels with particular concept for different questions are introduced for improving the prediction performance. Extensive experiments on three datasets have shown that the MQFKT approach achieves more precise prediction of student performance and better interpretation of the model.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Reasoning Beyond Bias: A Study on Counterfactual Prompting and Chain of Thought Reasoning

Kyle Moore,Jesse Roberts,Thao Pham,Douglas Fisher

2024-09-06

Abstract:Language models are known to absorb biases from their training data, leading to predictions driven by statistical regularities rather than semantic relevance. We investigate the impact of these biases on answer choice preferences in the Massive Multi-Task Language Understanding (MMLU) task. Our findings reveal that differences in learned regularities across answer options are predictive of model preferences and mirror human test-taking strategies. To address this issue, we introduce two novel methods: Counterfactual Prompting with Chain of Thought (CoT) and Counterfactual Prompting with Agnostically Primed CoT (APriCoT). We demonstrate that while Counterfactual Prompting with CoT alone is insufficient to mitigate bias, our novel Primed Counterfactual Prompting with CoT approach effectively reduces the influence of base-rate probabilities while improving overall accuracy. Our results suggest that mitigating bias requires a "System-2" like process and that CoT reasoning is susceptible to confirmation bias under some prompting methodologies. Our contributions offer practical solutions for developing more robust and fair language models.

Computation and Language,Artificial Intelligence
Enhancing Knowledge Tracing with Concept Map and Response Disentanglement

Soonwook Park,Donghoon Lee,Hogun Park

DOI: https://doi.org/10.1016/j.knosys.2024.112346

2024-08-23

Abstract:In the rapidly advancing realm of educational technology, it becomes critical to accurately trace and understand student knowledge states. Conventional Knowledge Tracing (KT) models have mainly focused on binary responses (i.e., correct and incorrect answers) to questions. Unfortunately, they largely overlook the essential information in students' actual answer choices, particularly for Multiple Choice Questions (MCQs), which could help reveal each learner's misconceptions or knowledge gaps. To tackle these challenges, we propose the Concept map-driven Response disentanglement method for enhancing Knowledge Tracing (CRKT) model. CRKT benefits KT by directly leveraging answer choices--beyond merely identifying correct or incorrect answers--to distinguish responses with different incorrect choices. We further introduce the novel use of unchosen responses by employing disentangled representations to get insights from options not selected by students. Additionally, CRKT tracks the student's knowledge state at the concept level and encodes the concept map, representing the relationships between them, to better predict unseen concepts. This approach is expected to provide actionable feedback, improving the learning experience. Our comprehensive experiments across multiple datasets demonstrate CRKT's effectiveness, achieving superior performance in prediction accuracy and interpretability over state-of-the-art models.

Artificial Intelligence,Machine Learning
Knowledge Tracing for Complex Problem Solving: Granular Rank-Based Tensor Factorization

Chunpai Wang,Shaghayegh Sahebi,Siqian Zhao,Peter Brusilovsky,Laura O. Moraes

DOI: https://doi.org/10.48550/arXiv.2210.09013

2022-10-06

Computers and Society

Abstract:Knowledge Tracing (KT), which aims to model student knowledge level and predict their performance, is one of the most important applications of user modeling. Modern KT approaches model and maintain an up-to-date state of student knowledge over a set of course concepts according to students' historical performance in attempting the problems. However, KT approaches were designed to model knowledge by observing relatively small problem-solving steps in Intelligent Tutoring Systems. While these approaches were applied successfully to model student knowledge by observing student solutions for simple problems, they do not perform well for modeling complex problem solving in students.M ost importantly, current models assume that all problem attempts are equally valuable in quantifying current student knowledge.However, for complex problems that involve many concepts at the same time, this assumption is deficient. In this paper, we argue that not all attempts are equivalently important in discovering students' knowledge state, and some attempts can be summarized together to better represent student performance. We propose a novel student knowledge tracing approach, Granular RAnk based TEnsor factorization (GRATE), that dynamically selects student attempts that can be aggregated while predicting students' performance in problems and discovering the concepts presented in them. Our experiments on three real-world datasets demonstrate the improved performance of GRATE, compared to the state-of-the-art baselines, in the task of student performance prediction. Our further analysis shows that attempt aggregation eliminates the unnecessary fluctuations from students' discovered knowledge states and helps in discovering complex latent concepts in the problems.
How deep is knowledge tracing?

Mohammad Khajah,Robert V. Lindsey,Michael C. Mozer

DOI: https://doi.org/10.48550/arXiv.1604.02416

IF: 14.4

2016-03-14

Artificial Intelligence

Abstract:In theoretical cognitive science, there is a tension between highly structured models whose parameters have a direct psychological interpretation and highly complex, general-purpose models whose parameters and representations are difficult to interpret. The former typically provide more insight into cognition but the latter often perform better. This tension has recently surfaced in the realm of educational data mining, where a deep learning approach to predicting students' performance as they work through a series of exercises---termed deep knowledge tracing or DKT---has demonstrated a stunning performance advantage over the mainstay of the field, Bayesian knowledge tracing or BKT. In this article, we attempt to understand the basis for DKT's advantage by considering the sources of statistical regularity in the data that DKT can leverage but which BKT cannot. We hypothesize four forms of regularity that BKT fails to exploit: recency effects, the contextualized trial sequence, inter-skill similarity, and individual variation in ability. We demonstrate that when BKT is extended to allow it more flexibility in modeling statistical regularities---using extensions previously proposed in the literature---BKT achieves a level of performance indistinguishable from that of DKT. We argue that while DKT is a powerful, useful, general-purpose framework for modeling student learning, its gains do not come from the discovery of novel representations---the fundamental advantage of deep learning. To answer the question posed in our title, knowledge tracing may be a domain that does not require `depth'; shallow models like BKT can perform just as well and offer us greater interpretability and explanatory power.

[Inhalation anesthesia in a completely closed circuit. A method of reducing the amount of anesthetic].

Do We Fully Understand Students' Knowledge States? Identifying and Mitigating Answer Bias in Knowledge Tracing

Interpretable Knowledge Tracing via Response Influence-based Counterfactual Reasoning

A Deeper Knowledge Tracking Model Integrating Cognitive Theory and Learning Behavior

A Temporal Convolutional Knowledge Tracing Model Integrating Forgetting Factors and Item Response Theory

Target Hierarchy-Guided Knowledge Tracing: Fine-Grained Knowledge State Modeling

Counterfactual Monotonic Knowledge Tracing for Assessing Students' Dynamic Mastery of Knowledge Concepts

A Question-centric Multi-experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing Models

Knowledge ontology enhanced model for explainable knowledge tracing

Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations

Programming Knowledge Tracing: A Comprehensive Dataset and A New Model

Interpretable Knowledge Tracing: Simple and Efficient Student Modeling with Causal Relations

Leveraging Pedagogical Theories to Understand Student Learning Process with Graph-based Reasonable Knowledge Tracing

Enhancing Deep Knowledge Tracing with Auxiliary Tasks

Beyond Right and Wrong: Mitigating Cold Start in Knowledge Tracing Using Large Language Model and Option Weight

Deep knowledge tracing with learning curves

Exploiting Multiple Question Factors for Knowledge Tracing

Reasoning Beyond Bias: A Study on Counterfactual Prompting and Chain of Thought Reasoning

Enhancing Knowledge Tracing with Concept Map and Response Disentanglement

Knowledge Tracing for Complex Problem Solving: Granular Rank-Based Tensor Factorization

How deep is knowledge tracing?