Abstract:Educational technology innovations leveraging large language models (LLMs) have shown the potential to automate the laborious process of generating and analysing textual content. While various innovations have been developed to automate a range of educational tasks (e.g., question generation, feedback provision, and essay grading), there are concerns regarding the practicality and ethicality of these innovations. Such concerns may hinder future research and the adoption of LLMs-based innovations in authentic educational contexts. To address this, we conducted a systematic scoping review of 118 peer-reviewed papers published since 2017 to pinpoint the current state of research on using LLMs to automate and support educational tasks. The findings revealed 53 use cases for LLMs in automating education tasks, categorised into nine main categories: profiling/labelling, detection, grading, teaching support, prediction, knowledge representation, feedback, content generation, and recommendation. Additionally, we also identified several practical and ethical challenges, including low technological readiness, lack of replicability and transparency, and insufficient privacy and beneficence considerations. The findings were summarised into three recommendations for future studies, including updating existing innovations with state-of-the-art models (e.g., GPT-3/4), embracing the initiative of open-sourcing models/systems, and adopting a human-centred approach throughout the developmental process. As the intersection of AI and education is continuously evolving, the findings of this study can serve as an essential reference point for researchers, allowing them to leverage the strengths, learn from the limitations, and uncover potential research opportunities enabled by ChatGPT and other generative AI models.

Beyond Search Engines: Can Large Language Models Improve Curriculum Development?

Large Language Models in Computer Science Education: A Systematic Literature Review

A Large Language Model Approach to Educational Survey Feedback Analysis

Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects

Large Language Models for Education: A Survey and Outlook

Adapting Large Language Models for Education: Foundational Capabilities, Potentials, and Challenges

Large Language Models As MOOCs Graders

Scaling Evidence-based Instructional Design Expertise through Large Language Models

LARGE LANGUAGE MODEL-BASED ARTIFICIAL INTELLIGENCE IN THE LANGUAGE CLASSROOM: PRACTICAL IDEAS FOR TEACHING

Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Concepts

An Exploration of Higher Education Course Evaluation by Large Language Models

Pedagogical Alignment of Large Language Models

Comparison of Large Language Models for Generating Contextually Relevant Questions

Comparison of Large Language Models in Generating Machine Learning Curricula in High Schools

Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions

Supervised Knowledge Makes Large Language Models Better In-context Learners

Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review

Evaluating and Optimizing Educational Content with Large Language Model Judgments

Automatically Generating CS Learning Materials with Large Language Models

Large Language Models: A Survey