Abstract:Personalized adaptive learning (PAL) stands out by closely monitoring individual students' progress and tailoring their learning paths to their unique knowledge and needs. A crucial technique for effective PAL implementation is knowledge tracing, which models students' evolving knowledge to predict their future performance. Recent advancements in deep learning have significantly enhanced knowledge tracing through Deep Knowledge Tracing (DKT). However, there is limited research on DKT for Science, Technology, Engineering, and Math (STEM) education at Historically Black Colleges and Universities (HBCUs). This study builds a comprehensive dataset to investigate DKT for implementing PAL in STEM education at HBCUs, utilizing multiple state-of-the-art (SOTA) DKT models to examine knowledge tracing performance. The dataset includes 352,148 learning records for 17,181 undergraduate students across eight colleges at Prairie View A&M University (PVAMU). The SOTA DKT models employed include DKT, DKT+, DKVMN, SAKT, and KQN. Experimental results demonstrate the effectiveness of DKT models in accurately predicting students' academic outcomes. Specifically, the SAKT and KQN models outperform others in terms of accuracy and AUC. These findings have significant implications for faculty members and academic advisors, providing valuable insights for identifying students at risk of academic underperformance before the end of the semester. Furthermore, this allows for proactive interventions to support students' academic progress, potentially enhancing student retention and graduation rates.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to solve the problem of how to accurately predict students' learning outcomes through Deep Knowledge Tracing (DKT) technology when implementing Personalized Adaptive Learning (PAL) in Historically Black Colleges and Universities (HBCUs). Specifically, the research focuses on the following aspects:
1. **Dataset construction**: Construct a comprehensive dataset for implementing PAL in STEM education in HBCUs. This dataset contains 352,148 learning records of 17,181 undergraduates from eight colleges at Prairie View A&M University (PVAMU).
2. **Model evaluation**: Utilize multiple state - of - the - art DKT models (including DKT, DKT +, DKVMN, SAKT, and KQN) to evaluate knowledge - tracing performance. These models aim to predict students' future learning performance by analyzing their past learning records.
3. **Performance comparison**: Through experimental results, compare the accuracy and AUC values of different DKT models in predicting students' academic achievements. The research finds that the SAKT and KQN models perform best in these two indicators.
4. **Practical application**: Explore the practical application value of these models, especially in identifying students with poor academic performance. This helps teachers and academic advisors take intervention measures in advance to support students' academic progress, thereby increasing student retention and graduation rates.
### Research background
Personalized Adaptive Learning (PAL), by closely monitoring each student's learning progress and customizing the learning path according to their unique knowledge and needs, has become an important method in modern education. Knowledge Tracing (KT) is a key technology for achieving effective PAL. It predicts students' future performance by modeling the evolution of their knowledge. In recent years, the development of deep - learning technology has significantly enhanced the effect of knowledge tracing, forming Deep Knowledge Tracing (DKT). However, in STEM education in HBCUs, there are relatively few studies on DKT.
### Main contributions
1. **Dataset construction**: This is the first DKT dataset for HBCUs, exploring the potential of PAL. This dataset enables researchers to capture, analyze, and interpret data related to STEM education in HBCUs and implement PAL using advanced knowledge - tracing techniques.
2. **Model evaluation**: Evaluate the effectiveness and limitations of current state - of - the - art DKT models in predicting student performance and early identification of at - risk students. The research not only fills the gap in DKT research and application in the specific educational context of HBCUs but also broadens the scope of DKT application, providing valuable implementation strategies and implications for future research and practice.
### Experimental design
1. **Data collection and preparation**: Four years of data were collected from PVAMU's Student Information System (SIS), covering five elements: academic year, general ID, course subject, course level, and pass/fail grades.
2. **Data pre - processing**: Exclude records with missing grades, incomplete courses, or non - graded courses; convert categorical data into numerical format, and use label encoding to convert course subjects and levels into integers representing students' knowledge and skills.
3. **Model training and testing**: Divide the data into a training set and a test set, where the data of the first three years are used for training and the data of the fourth year are used for testing. The training set includes three subsets: data from the College of Engineering (COE), data from the College of Engineering and the College of Arts and Sciences (COE + COAS), and data from the entire university (UNIV).
### Experimental results
1. **Performance comparison**: In tests in multiple engineering departments, the KQN model performs excellently, with an average AUC of 0.6965, an accuracy rate of 0.7996, and an F1 score of 0.8834. The SAKT and DKVMN models also show competitiveness, achieving good results in AUC, accuracy rate, and F1 score respectively.
2. **Practical application**: The high prediction accuracy of these models helps teachers and academic advisors identify students who may have poor academic performance in advance, so as to take timely intervention measures to support students' academic progress.
### Conclusion
This research shows the potential of implementing personalized adaptive learning in STEM education in HBCUs by constructing a comprehensive dataset and evaluating multiple state - of - the - art DKT models. The research results not only verify the effectiveness of these models in predicting students' academic achievements but also provide for future teaching.