Abstract:Instructor's feedback plays a critical role in students' development of conceptual understanding and reasoning skills. However, grading student written responses and providing personalized feedback can take a substantial amount of time. In this study, we explore using GPT-3.5 to write feedback to student written responses to conceptual questions with prompt engineering and few-shot learning techniques. In stage one, we used a small portion (n=20) of the student responses on one conceptual question to iteratively train GPT. Four of the responses paired with human-written feedback were included in the prompt as examples for GPT. We tasked GPT to generate feedback to the other 16 responses, and we refined the prompt after several iterations. In stage two, we gave four student researchers the 16 responses as well as two versions of feedback, one written by the authors and the other by GPT. Students were asked to rate the correctness and usefulness of each feedback, and to indicate which one was generated by GPT. The results showed that students tended to rate the feedback by human and GPT equally on correctness, but they all rated the feedback by GPT as more useful. Additionally, the successful rates of identifying GPT's feedback were low, ranging from 0.1 to 0.6. In stage three, we tasked GPT to generate feedback to the rest of the student responses (n=65). The feedback was rated by four instructors based on the extent of modification needed if they were to give the feedback to students. All the instructors rated approximately 70% of the feedback statements needing only minor or no modification. This study demonstrated the feasibility of using Generative AI as an assistant to generating feedback for student written responses with only a relatively small number of examples. An AI assistance can be one of the solutions to substantially reduce time spent on grading student written responses.

Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions

Automated Essay Feedback Generation and Its Impact on Revision

Evaluating Language Models for Generating and Judging Programming Feedback

Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences

AI-generated feedback on writing: insights into efficacy and ENL student preference

Generating Feedback-Ladders for Logical Errors in Programming using Large Language Models

Automated Feedback in Math Education: A Comparative Analysis of LLMs for Open-Ended Responses

Generative AI as a Tool for Enhancing Reflective Learning in Students

Improving the Validity of Automatically Generated Feedback via Reinforcement Learning

Students' Perceptions and Preferences of Generative Artificial Intelligence Feedback for Programming

Improving Code Generation by Training with Natural Language Feedback

A large language model-assisted education tool to provide feedback on open-ended responses

"My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays

Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge

Using Large Language Models to Provide Explanatory Feedback to Human Tutors

Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation

Feedback, feedback-on-feedback and re-feedback: effects of written dialogic peer feedback on English as a foreign language writing

Integrating AI for Enhanced Feedback in Translation Revision- A Mixed-Methods Investigation of Student Engagement

From Model-centered to Human-Centered: Revision Distance as a Metric for Text Evaluation in LLMs-based Applications

Using teacher-supported peer feedback to develop student feedback literacy in L2 writing: An instructional model

Exploring Generative AI assisted feedback writing for students' written responses to a physics conceptual question with prompt engineering and few-shot learning