Befriending ChatGPT and other superchatbots: An AI-integrated take-home assessment preserving integrity

P G Kubendran Amos
2023-06-03
Abstract:With the launch of ChatGPT, serious concerns have reasonably been raised of its ill-effect on the integrity of remote take-home exams. By way of mitigating the concern, in this study, a rather straightforward Artificial-Intelligence (AI)-integrated take-home assessment technique is proposed, and the outcome of its practice is discussed. Despite involving AI, in the form of ChatGPT, the assessment adheres to the convention of posing questions invoking critical thinking and problem solving skills. However, AI is characteristically integrated in this assessment by instructing the learners to employ ChatGPT as one of the primary sources. The learners are directed to report the use of ChatGPT by including both the prompts and its responses, before expressing their thoughts on AI-generated answers and their own concluding statement. These three characteristic components of the present techniques -- the handling of ChatGPT through the prompts, comments on the AI-responses and the concluding thoughts -- are evaluated to gauge the learning. The proposed assessment was assigned as a take-home group activity for a batch of seventy eight students, divided into thirteen groups. Despite addressing the same questions, there was no significant overlap in the answers. Moreover, a wide range of approaches were adopted by the groups in handling ChatGPT, which in-turn rendered different responses, ultimately drawing distinct answers. Besides preventing the undesired use of ChatGPT by explicitly integrating it, the proposed assessment seemingly helped the learners question the accuracy of its responses. This self-realised skepticism can be expected to curtail blatant malpractices involving ChatGPT in the long run.
Physics Education
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: How to effectively integrate advanced chatbots like ChatGPT into take - home remote exams while maintaining academic integrity. With the launch of ChatGPT, people have expressed reasonable concerns about its possible negative impact on the academic integrity of take - home remote exams. For this reason, the author proposes a simple take - home assessment technique for artificial intelligence (AI) integration and discusses the practical results. This technique not only allows students to use ChatGPT as one of the main sources, but also requires students to report their use of ChatGPT, including the prompts used and ChatGPT's responses, and express their views on AI - generated answers and their own conclusions. In this way, the assessment aims to measure students' ability to handle ChatGPT, their critical thinking about AI responses, and their final conclusion - forming ability, while ensuring the integrity and effectiveness of the assessment. Specifically, the paper explores the following aspects: - How to design an assessment method that can take advantage of AI tools while preventing their improper use. - By allowing students to explicitly use ChatGPT and requiring them to record and comment on ChatGPT's output, to improve students' critical thinking ability and the rational use of AI tools. - To assess the effectiveness of this method in practical applications, including students' participation, the diversity of answers, and students' critical analysis ability of ChatGPT's output. In short, the main purpose of this paper is to explore an innovative educational assessment method to adapt to the development of artificial intelligence technology while maintaining the fairness and quality of education.