Better by you, better than me, chatgpt3 as writing assistance in students essays

Zeljana Basic,Ana Banovac,Ivana Kruzic,Ivan Jerkovic
DOI: https://doi.org/10.1057/s41599-023-02269-7
2023-10-31
Abstract:Aim: To compare students' essay writing performance with or without employing ChatGPT-3 as a writing assistant tool. Materials and methods: Eighteen students participated in the study (nine in control and nine in the experimental group that used ChatGPT-3). We scored essay elements with grades (A-D) and corresponding numerical values (4-1). We compared essay scores to students' GPTs, writing time, authenticity, and content similarity. Results: Average grade was C for both groups; for control (2.39, SD=0.71) and for experimental (2.00, SD=0.73). None of the predictors affected essay scores: group (P=0.184), writing duration (P=0.669), module (P=0.388), and GPA (P=0.532). The text unauthenticity was slightly higher in the experimental group (11.87%, SD=13.45 to 9.96%, SD=9.81%), but the similarity among essays was generally low in the overall sample (the Jaccard similarity index ranging from 0 to 0.054). In the experimental group, AI classifier recognized more potential AI-generated texts. Conclusions: This study found no evidence that using GPT as a writing tool improves essay quality since the control group outperformed the experimental group in most parameters.
Artificial Intelligence,Computers and Society,Human-Computer Interaction
What problem does this paper attempt to address?
The paper aims to explore whether using ChatGPT-3 as a writing assistance tool can improve the quality of students' essays, reduce writing time, and affect the authenticity of the text. The study evaluates the impact of ChatGPT-3 on students' writing performance, writing time, and text originality by comparing the performance of students in the experimental group (using ChatGPT-3 for writing assistance) with those in the control group (traditional writing methods) in writing tasks. The results indicate that using ChatGPT-3 did not significantly enhance the quality of students' essays, and the writing scores of students in the experimental group were slightly lower than those in the control group. Additionally, the time required to complete the tasks was similar for both groups, while the non-originality of the texts in the experimental group was slightly higher, but overall, the similarity between the essays of the two groups was low. These findings suggest that under current conditions, the effectiveness of ChatGPT-3 as a writing assistance tool for students is limited and may not significantly improve students' writing performance.