Abstract:Programming courses can be challenging for first year university students, especially for those without prior coding experience. Students initially struggle with code syntax, but as more advanced topics are introduced across a semester, the difficulty in learning to program shifts to learning computational thinking (e.g., debugging strategies). This study examined the relationships between students' rate of programming errors and their grades on two exams. Using an online integrated development environment, data were collected from 280 students in a Java programming course. The course had two parts. The first focused on introductory procedural programming and culminated with exam 1, while the second part covered more complex topics and object-oriented programming and ended with exam 2. To measure students' programming abilities, 51095 code snapshots were collected from students while they completed assignments that were autograded based on unit tests. Compiler and runtime errors were extracted from the snapshots, and three measures -- Error Count, Error Quotient and Repeated Error Density -- were explored to identify the best measure explaining variability in exam grades. Models utilizing Error Quotient outperformed the models using the other two measures, in terms of the explained variability in grades and Bayesian Information Criterion. Compiler errors were significant predictors of exam 1 grades but not exam 2 grades; only runtime errors significantly predicted exam 2 grades. The findings indicate that leveraging Error Quotient with multiple error types (compiler and runtime) may be a better measure of students' introductory programming abilities, though still not explaining most of the observed variability.

Data-Driven Investigation into Variants of Code Writing Questions.

Unique Exams: Designing assessments for integrity and fairness

Code Interviews: Design and Evaluation of a More Authentic Assessment for Introductory Programming Assignments

An Approach to Detect Abnormal Submissions for CodeWorkout Dataset

Machine learning based approach to exam cheating detection

A Format for a Plagiarism-Proof Online Examination for Calculus and Linear Algebra Using Microsoft Excel

SimGrade: Using Code Similarity Measures for More Accurate Human Grading

Anti-cheating Online Exams by Minimizing the Cheating Gain

Keystroke Dynamics Against Academic Dishonesty in the Age of LLMs

Let's Ask Students About Their Programs, Automatically

A Data Mining Approach for Detecting Collusion in Unproctored Online Exams

Plagiarism deterrence for introductory programming

Question-Score Identity Detection (Q-SID): A Statistical Algorithm to Detect Collusion Groups with Error Quantification from Exam Question Scores

Comparison of Three Programming Error Measures for Explaining Variability in CS1 Grades

Automated Plagiarism Detection for Computer Programming Exercises Based on Patterns of Resubmission

Are Exam Questions Known in Advance? Using Local Dependence to Detect Cheating

A Visual Analytics Approach to Facilitate the Proctoring of Online Exams

Cheat sites and artificial intelligence usage in online introductory physics courses: what is the extent and what effect does it have on assessments?

Automatic question generation for propositional logical equivalences

Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation

DiffGen: Automated Unit Test Generation for Regression Testing