Can an AI-tool grade assignments in an introductory physics course?

Gerd Kortemeyer
DOI: https://doi.org/10.1103/PhysRevPhysEducRes.19.020163
2023-04-22
Abstract:Problem solving is an integral part of any physics curriculum, and most physics instructors would likely agree that the associated learner competencies are best assessed by considering the solution path: not only the final solution matters, but also how the learner arrived there. Unfortunately, providing meaningful feedback on written derivations is much more labor and resource intensive than only grading the outcome: currently, the latter can be done by computer, while the former involves handwritten solutions that need to be graded by humans. This exploratory study proposes an AI-assisted workflow for grading written physics-problem solutions, and it evaluates the viability of the actual grading step using GPT-4. It is found that the AI-tool is capable of providing feedback that can be helpful in formative assessment scenarios, but that for summative scenarios, particularly those that are high-stakes, it should only be used for an initial round of grading that sorts and flags solution approaches.
Physics Education
What problem does this paper attempt to address?
The problem this paper attempts to address is: Can artificial intelligence tools be used to grade assignments in introductory physics courses? Specifically, the authors explore the feasibility of using AI tools (particularly GPT-4) to grade written solutions to physics problems. The study finds that AI tools can provide feedback that is helpful for formative assessment, but for summative assessment, especially in high-stakes situations, it is recommended to use them only for preliminary screening and marking of solutions.