Understanding Revision Behavior in Adaptive Writing Support Systems for Education

Luca Mouchel,Thiemo Wambsganss,Paola Mejia-Domenzain,Tanja Käser

DOI: https://doi.org/10.5281/zenodo.8115766

2023-06-17

Abstract:Revision behavior in adaptive writing support systems is an important and relatively new area of research that can improve the design and effectiveness of these tools, and promote students' self-regulated learning (SRL). Understanding how these tools are used is key to improving them to better support learners in their writing and learning processes. In this paper, we present a novel pipeline with insights into the revision behavior of students at scale. We leverage a data set of two groups using an adaptive writing support tool in an educational setting. With our novel pipeline, we show that the tool was effective in promoting revision among the learners. Depending on the writing feedback, we were able to analyze different strategies of learners when revising their texts, we found that users of the exemplary case improved over time and that females tend to be more efficient. Our research contributes a pipeline for measuring SRL behaviors at scale in writing tasks (i.e., engagement or revision behavior) and informs the design of future adaptive writing support systems for education, with the goal of enhancing their effectiveness in supporting student writing. The source code is available at <a class="link-external link-https" href="https://github.com/lucamouchel/Understanding-Revision-Behavior" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Information Retrieval

What problem does this paper attempt to address?

The paper primarily explores the application of adaptive writing support systems in educational settings and attempts to understand students' revision behaviors when using these tools. Specifically, the researchers developed a new analysis pipeline to measure and visualize students' self-regulated learning (SRL) behaviors on a large scale, particularly their engagement and revision behaviors in writing tasks. By analyzing the performance differences between two groups of students before and after using the adaptive feedback tool, the study found:

1. **Differences in Revision Strategies**: Students who received adaptive feedback exhibited more revision steps during the revision process, but each revision involved fewer changes; conversely, students who did not receive feedback revised less frequently but made more substantial changes each time.
2. **Gender Differences**: Female participants demonstrated higher revision efficiency than males, especially in the absence of adaptive feedback, where females showed even higher revision efficiency.
3. **Changes in Engagement**: Starting from the second piece of work, all students showed a decline in revision frequency, number of revisions, and revision time, indicating a decrease in interest or engagement with the task as the experiment progressed.

The study also proposed some future research directions, such as using cluster analysis to further understand different revision behavior patterns. Overall, this research provides valuable insights into how adaptive writing support systems influence students' revision behaviors and offers references for designing more effective educational tools.