Exploring AI-Generated Text in Student Writing: How Does AI Help?

David James Woo,Hengky Susanto,Chi Ho Yeung,Kai Guo,April Ka Yeng Fung

2024-01-01

Abstract:English as foreign language_EFL_students' use of text generated from artificial intelligence_AI_natural language generation_NLG_tools may improve their writing quality. However, it remains unclear to what extent AI-generated text in these students' writing might lead to higher-quality writing. We explored 23 Hong Kong secondary school students' attempts to write stories comprising their own words and AI-generated text. Human experts scored the stories for dimensions of content, language and organization. We analyzed the basic organization and structure and syntactic complexity of the stories' AI-generated text and performed multiple linear regression and cluster analyses. The results show the number of human words and the number of AI-generated words contribute significantly to scores. Besides, students can be grouped into competent and less competent writers who use more AI-generated text or less AI-generated text compared to their peers. Comparisons of clusters reveal some benefit of AI-generated text in improving the quality of both high-scoring students' and low-scoring students' writing. The findings can inform pedagogical strategies to use AI-generated text for EFL students' writing and to address digital divides. This study contributes designs of NLG tools and writing activities to implement AI-generated text in schools.

Computation and Language,Artificial Intelligence,Computers and Society

What problem does this paper attempt to address?

The paper attempts to address the issue of exploring the effects of using AI-generated text by English as a Foreign Language (EFL) learners during the writing process and its impact on overall writing quality. Specifically, the study focuses on the following aspects:

1. **The use of AI-generated text by EFL students**: The study analyzed how 23 secondary school students in Hong Kong integrated their own writing with AI-generated text when composing stories, and examined the basic structure, organizational form, and syntactic complexity of these texts.

2. **The impact of AI-generated text on writing quality**: Experts scored each story based on content, language, and organization. Through multiple linear regression and cluster analysis methods, the study found that both the number of human-generated words and AI-generated words significantly affected the writing scores.

3. **Differences among students of different levels**: Through cluster analysis, the study revealed different patterns exhibited by high-scoring and low-scoring students in utilizing AI-generated text, and pointed out that this technology helps improve the writing performance of different types of learners.

In summary, this paper aims to explore how to effectively use AI tools to assist EFL students in their writing activities and proposes differentiated teaching strategies to help students better apply AI-generated text to complete writing tasks.