Abstract:Background and Context In today's tech-driven world, programming courses are crucial. Yet, teaching programming is challenging, leading to high student failure rates. Understanding student learning patterns is key, but there's a lack of research utilizing tools to automatically collect and analyze interaction data for insights into student performance and behaviors. Objectives Study aims to compare problem-solving behaviors of novice and competent programmers during coding tests, identifying patterns and exploring relationships with program correctness. Method We built an online system with programming challenges to collect behavior data from novice and competent programmers. Our system analyzed data using various metrics to explore behavior-program correctness relationships. Findings Analysis showed distinct problem-solving behavior patterns. Competent programmers had fewer syntax errors, spent less time fixing bugs, and had higher program correctness. Novices made more syntax errors and spent more time fixing coding errors. Both groups used tabs for code structure, but competent programmers introduced unfamiliar variables more often and commented on them afterward. Emphasizing iterative revisions and active engagement enhances problem-solving skills and programming proficiency. Radar charts are effective for identifying improvement areas in teaching programming. The relationship between behavior and program correctness was positively correlated for competent programmers but not novices. Implications Study findings have implications for programming education. Radar charts help teachers identify course improvement areas. Novices can learn from competent programmers' behavior. Instructors should encourage continuous skill improvement through revisions and engagement. Identified unfamiliar programming aspects offer insights for targeted learning.

C-Pack of IPAs: A C90 Program Benchmark of Introductory Programming Assignments

InvAASTCluster: On Applying Invariant-Based Program Clustering to Introductory Programming Assignments

BRAFAR: Bidirectional Refactoring, Alignment, Fault Localization, and Repair for Programming Assignments

PROGpedia: Collection of source-code submitted to introductory programming assignments

Automated Program Repair for Introductory Programming Assignments Via Bidirectional Refactoring

Repairing Bugs in Python Assignments Using Large Language Models

Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignments

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Analyzing Novice and Competent Programmers' Problem-Solving Behaviors Using an Automated Evaluation System

Automatic Grading and Feedback using Program Repair for Introductory Programming Courses

Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming

FAPR: Fast and Accurate Program Repair for Introductory Programming Courses

PaCon: a Symbolic Analysis Approach for Tactic-Oriented Clustering of Programming Submissions.

Let's Ask Students About Their Programs, Automatically

Data-Driven Feedback Generation for Introductory Programming Exercises

OctoPack: Instruction Tuning Code Large Language Models

Benchmarking Educational Program Repair

A Picture Is Worth a Thousand Words: Exploring Diagram and Video-Based OOP Exercises to Counter LLM Over-Reliance

Automated Feedback Generation for Introductory Programming Assignments

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students

Adaptive Scaffolding in Block-Based Programming via Synthesizing New Tasks as Pop Quizzes