How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course

Andreas Scholl,Natalie Kiesler
2024-07-30
Abstract:This research paper contributes to the computing education research community's understanding of Generative AI (GenAI) in the context of introductory programming, and specifically, how students utilize related tools, such as ChatGPT. An increased understanding of students' use is mandatory for educators and higher education institutions, as GenAI is here to stay, and its performance is likely to improve rapidly in the near future. Learning about students' use patterns is not only crucial to support their learning, but to develop adequate forms of instruction and assessment. With the rapid advancement of AI, its broad availability, and ubiquitous presence in educational environments, elaborating how AI can enhance learning experiences, especially in courses such as introductory programming is important. To date, most studies have focused on the educator's perspective on GenAI, its performance, characteristics, and limitations. However, the student perspective, and how they actually use GenAI tools in course contexts, has not been subject to a great number of studies. Therefore, this study is guided by the following research questions: (1) What do students report on their use pattern of ChatGPT in the context of introductory programming exercises? and (2) How do students perceive ChatGPT in the context of introductory programming exercises? To address these questions, computing students at a large German university were asked to solve programming tasks with the assistance of ChatGPT as part of their introductory programming course. Students (n=298) provided information regarding the use of ChatGPT, and their evaluation of the tool via an online survey. This research provides a comprehensive evaluation of ChatGPT-3.5's application by novice programmers in a higher education context...
Artificial Intelligence
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how students use and experience generative artificial intelligence tools (such as ChatGPT) in an introductory computer programming course. Specifically, the research aims to answer two main research questions: 1. What are the behavior patterns that students report when using ChatGPT to solve programming exercises? 2. How do students perceive ChatGPT when solving programming exercises? To answer these questions, the researchers designed a survey in which students majoring in computer science at a large university in Germany were asked to use ChatGPT - 3.5 when solving programming tasks, and their feedback was collected through an online survey. This study provides a detailed assessment of students' use of ChatGPT in the curriculum, revealing that although students widely adopt generative AI, their ways of using it vary significantly, ranging from accepting generated solutions to dynamic and critical participation. These findings are of great significance for educators to design guiding principles or teaching forms for using generative AI tools in the classroom.