Leveraging generative artificial intelligence to simulate student learning behavior

Songlin Xu,Xinyu Zhang

2023-10-30

Abstract:Student simulation presents a transformative approach to enhance learning outcomes, advance educational research, and ultimately shape the future of effective pedagogy. We explore the feasibility of using large language models (LLMs), a remarkable achievement in AI, to simulate student learning behaviors. Unlike conventional machine learning based prediction, we leverage LLMs to instantiate virtual students with specific demographics and uncover intricate correlations among learning experiences, course materials, understanding levels, and engagement. Our objective is not merely to predict learning outcomes but to replicate learning behaviors and patterns of real students. We validate this hypothesis through three experiments. The first experiment, based on a dataset of N = 145, simulates student learning outcomes from demographic data, revealing parallels with actual students concerning various demographic factors. The second experiment (N = 4524) results in increasingly realistic simulated behaviors with more assessment history for virtual students modelling. The third experiment (N = 27), incorporating prior knowledge and course interactions, indicates a strong link between virtual students' learning behaviors and fine-grained mappings from test questions, course materials, engagement and understanding levels. Collectively, these findings deepen our understanding of LLMs and demonstrate its viability for student simulation, empowering more adaptable curricula design to enhance inclusivity and educational effectiveness.

Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the issue of how to utilize large language models (LLMs) to simulate student learning behaviors, thereby improving educational outcomes, advancing educational research, and ultimately shaping effective teaching methods. Specifically, the focus of the paper is not merely on predicting learning outcomes, but on revealing the complex relationships between learning experiences, course materials, levels of understanding, and engagement by simulating the learning behaviors and patterns of real students.

The paper validates this hypothesis through three experiments:
1. **Experiment 1**: Based on a dataset containing 145 students, it simulates the impact of different demographic characteristics on student learning outcomes.
2. **Experiment 2**: Based on a dataset containing 4524 students, it incorporates assessment history into the simulation process. The results show that as more assessment history data is included, the simulated student learning behaviors become more realistic.
3. **Experiment 3**: Based on a dataset containing 27 students, it not only considers demographic information but also the interaction between students and course materials, including students' eye-tracking data, pre-test and post-test scores, to simulate student learning experiences and outcomes in a more detailed manner.

The results of these experiments deepen our understanding of LLMs and demonstrate their feasibility and effectiveness in student simulation, thereby helping to design more adaptive and inclusive curricula to enhance educational outcomes.