To what extent is ChatGPT useful for language teacher lesson plan creation?

Alex Dornburg,Kristin Davin

2024-04-25

Abstract:The advent of generative AI models holds tremendous potential for aiding teachers in the generation of pedagogical materials. However, numerous knowledge gaps concerning the behavior of these models obfuscate the generation of research-informed guidance for their effective usage. Here we assess trends in prompt specificity, variability, and weaknesses in foreign language teacher lesson plans generated by zero-shot prompting in ChatGPT. Iterating a series of prompts that increased in complexity, we found that output lesson plans were generally high quality, though additional context and specificity to a prompt did not guarantee a concomitant increase in quality. Additionally, we observed extreme cases of variability in outputs generated by the same prompt. In many cases, this variability reflected a conflict between 20th century versus 21st century pedagogical practices. These results suggest that the training of generative AI models on classic texts concerning pedagogical practices may represent a currently underexplored topic with the potential to bias generated content towards teaching practices that have been long refuted by research. Collectively, our results offer immediate translational implications for practicing and training foreign language teachers on the use of AI tools. More broadly, these findings reveal the existence of generative AI output trends that have implications for the generation of pedagogical materials across a diversity of content areas.

Computers and Society,Artificial Intelligence,Computation and Language,Human-Computer Interaction

What problem does this paper attempt to address?

The paper primarily explores the application effects and limitations of ChatGPT in lesson plan design for language teachers. Specifically, the study evaluates ChatGPT's ability to generate lesson plans through a series of prompts with varying complexity. The research found that although the overall quality of the generated lesson plans was high, increasing the specificity of the prompts did not always result in a corresponding improvement in quality. Additionally, there was significant variability in the lesson plans generated from the same prompt, and this variability sometimes reflected conflicts between 20th and 21st-century teaching methods. The findings suggest that the classic texts used to train generative AI models may lead to content that is biased towards teaching practices that have been refuted by modern research. Therefore, the study emphasizes the importance of understanding and mastering how to effectively use prompt engineering and provides practical guidance for foreign language teachers on using AI tools. Overall, the research aims to assess the effectiveness and limitations of zero-shot prompting in lesson plan design for language teachers.