Analyzing Novice and Competent Programmers' Problem-Solving Behaviors Using an Automated Evaluation System

Yung-Ting Chuang,Hsin-Yu Chang
DOI: https://doi.org/10.1016/j.scico.2024.103138
IF: 1.039
2024-05-10
Science of Computer Programming
Abstract:Background and Context In today's tech-driven world, programming courses are crucial. Yet, teaching programming is challenging, leading to high student failure rates. Understanding student learning patterns is key, but there's a lack of research utilizing tools to automatically collect and analyze interaction data for insights into student performance and behaviors. Objectives Study aims to compare problem-solving behaviors of novice and competent programmers during coding tests, identifying patterns and exploring relationships with program correctness. Method We built an online system with programming challenges to collect behavior data from novice and competent programmers. Our system analyzed data using various metrics to explore behavior-program correctness relationships. Findings Analysis showed distinct problem-solving behavior patterns. Competent programmers had fewer syntax errors, spent less time fixing bugs, and had higher program correctness. Novices made more syntax errors and spent more time fixing coding errors. Both groups used tabs for code structure, but competent programmers introduced unfamiliar variables more often and commented on them afterward. Emphasizing iterative revisions and active engagement enhances problem-solving skills and programming proficiency. Radar charts are effective for identifying improvement areas in teaching programming. The relationship between behavior and program correctness was positively correlated for competent programmers but not novices. Implications Study findings have implications for programming education. Radar charts help teachers identify course improvement areas. Novices can learn from competent programmers' behavior. Instructors should encourage continuous skill improvement through revisions and engagement. Identified unfamiliar programming aspects offer insights for targeted learning.
computer science, software engineering
What problem does this paper attempt to address?