Online Programming Exams - an Experience Report

Sean Russell,Simon Caton,Brett A. Becker

DOI: https://doi.org/10.1145/3587102.3588829

2023-01-01

Abstract:When seeking to maximise the authenticity of assessment in programming courses it makes sense to provide students with practical programming problems to solve in an environment that is close to real software development practice, i.e., online, open book, and using their typical development environment. This creates an assessment environment that should afford students sufficient opportunities to evidence what they have learned, but also creates practical challenges in terms of academic integrity, flexibility in the automated grading process, and assumptions surrounding how the student may attempt to solve the problems both in terms of correct and incorrect solutions. In this experience report, we outline two independently observed cohorts of students sitting the same Java programming exam, with different weights, over three years. This is undertaken as a reflective exercise in order to derive a series of recommendations and retrospectively obvious pitfalls to act as guidance for educators considering online programming exams for large (i.e. n > 150) introductory programming courses. After discussing our assessment methodology, we provide 4 high-level observations and centre a set of recommendations around these to aid practitioners in their assessment design.