Abstract:Background: Programming skills are advantageous to navigate today's society, so it is important to teach them to students. However, failure rates for programming courses are high, and especially students who fall behind early in introductory programming courses tend to stay behind. Objective: To catch these students as early as possible, we aim to develop an early warning system, so we can offer the students support, for example, in the form of syntax drill-and-practice exercises. Method: To develop the early warning system, we assess different cognitive skills of students of an introductory programming course. On several points in time over the course, students complete tests that measure their ability to develop a mental model of programming, language skills, attention, and fluid intelligence. Then, we evaluated to what extent these skills predict whether students acquire programming skills. Additionally, we assess how syntax drill-and-practice exercises improve how students acquire programming skill. Findings: Most of the cognitive skills can predict whether students acquire programming skills to a certain degree. Especially the ability to develop an early mental model of programming and language skills appear to be relevant. Fluid intelligence also shows predictive power, but appears to be comparable with the ability to develop a mental model. Furthermore, we found a significant positive effect of the syntax drill-and-practice exercises on the success of a course. Implications: Our first suggestion of an early warning system consists of few, easy-to-apply tests that can be integrated in programming courses or applied even before a course starts. Thus, with the start of a programming course, students who are at high risk of failing can be identified and offered support, for example, in the form of syntax drill-and-practice exercises to help students to develop programming skills.

Ability-training-oriented automated assessment in introductory programming course

Autolep: an Automated Learning and Examination System for Programming and Its Application in Programming Course

The Recent Development of Automated Programming Assessment

Assessment of Programming Language Learning Based on Peer Code Review Model: Implementation and Experience Report

Using an automated assessment tool to explore difficulties of middle school students in introductory programming

Analyzing Novice and Competent Programmers' Problem-Solving Behaviors Using an Automated Evaluation System

Real-time learning analytics for C programming language courses

ProgEdu4Web: An automated assessment tool for motivating the learning of web programming course

Automatic grading system for C programming examination oriented to assessment of comprehensive practical abilities

Formative Assessment of Programming Language Learning Based on Peer Code Review: Implementation and Experience Report

Automated Assessment in Computer Science Education: A State-of-the-Art Review

Teaching Python programming with automatic assessment and feedback provision

An Automated System for Interactively Learning Software Testing

PowerGrader: Automating Code Assessment Based on PowerShell for Programming Courses.

Automated assessment system for programming courses: a case study for teaching data structures and algorithms

OPTES: A Tool for Behavior-based Student Programming Progress Estimation.

Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study

Building an Effective Automated Assessment System for C/C++ Introductory Programming Courses in ODL Environment

Building a Comprehensive Automated Programming Assessment System

Toward Finding and Supporting Struggling Students in a Programming Course with an Early Warning System

How Are Students' Programming Skills Developed