Abstract:The recent release of very large language models such as PaLM and GPT-4 has made an unprecedented impact in the popular media and public consciousness, giving rise to a mixture of excitement and fear as to their capabilities and potential uses, and shining a light on natural language processing research which had not previously received so much attention. The developments offer great promise for education technology, and in this paper we look specifically at the potential for incorporating large language models in AI-driven language teaching and assessment systems. We consider several research areas and also discuss the risks and ethical considerations surrounding generative AI in education technology for language learners. Overall we find that larger language models offer improvements over previous models in text generation, opening up routes toward content generation which had not previously been plausible. For text generation they must be prompted carefully and their outputs may need to be reshaped before they are ready for use. For automated grading and grammatical error correction, tasks whose progress is checked on well-known benchmarks, early investigations indicate that large language models on their own do not improve on state-of-the-art results according to standard evaluation metrics. For grading it appears that linguistic features established in the literature should still be used for best performance, and for error correction it may be that the models can offer alternative feedback styles which are not measured sensitively with existing methods. In all cases, there is work to be done to experiment with the inclusion of large language models in education technology for language learners, in order to properly understand and report on their capacities and limitations, and to ensure that foreseeable risks such as misinformation and harmful bias are mitigated.

Developing a framework to re-design writing assignment assessment for the era of Large Language Models

Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI

Applying Large Language Models for Automated Essay Scoring for Non-Native Japanese

Embracing AI in Education: Understanding the Surge in Large Language Model Use by Secondary Students

Using Large Language Models for Automated Grading of Student Writing about Science

On the application of Large Language Models for language teaching and assessment technology

The Influence of GenAI on the Effectiveness of Argumentative Writing in Higher Education: Evidence from a Quasi-Experimental Study in China

Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Course

Prompting Higher Education Towards AI-Augmented Teaching and Learning Practice

ChatGPT and large language models in academia: opportunities and challenges

Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review

AI-generated feedback on writing: insights into efficacy and ENL student preference

Academic integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond

Exploring the Role of Artificial Intelligence in Facilitating Assessment of Writing Performance in Second Language Learning

Large Language Models As MOOCs Graders

AI-writing tools in education: if you can't beat them, join them

Enhancing university level English proficiency with generative AI: Empirical insights into automated feedback and learning outcomes

Large Language Models in Student Assessment: Comparing ChatGPT and Human Graders

The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models

On the assessment of generative AI in modeling tasks: an experience report with ChatGPT and UML

Harnessing LLMs for multi-dimensional writing assessment: Reliability and alignment with human judgments