Abstract:Specifications grading is an alternative grading system that has been used with increasing frequency in higher education. Since first introduced by Linda Nilson in 2014, more than 91 publications on the design and implementation of specifications grading systems have been published. This work presents a systematic review of the current literature to analyze the variety of ways specifications grading systems are executed, including the diverse design and implementation considerations, as well as to present and discuss emergent themes. We analyzed 91 publications and present their relevant findings in the results. The following databases were last searched on October 5, 2024, for publications: IEEE Xplore, ACS Publications, ASEE PEER, PER, Scopus, ERIC, ACM, ScienceDirect, and Web of Science. All peer-reviewed journal articles, conference proceedings, and book chapters that implemented at least two structural features of specifications grading in an undergraduate or graduate course were included in this review. Theses, dissertations, conference abstracts, posters, workshops, blogs, opinion pieces, social media exchanges, and content provided on websites were not included. Additionally, reports of specifications grading systems in K-12 courses or those that only presented the design and/or implementation of less than two structural features of the grading system were similarly excluded. Our findings from the literature reveal that the following themes emerge from educators who use specifications grading: time investment, academic performance, and student reactions. This review provides a resource for those interested in exploring this alternative grading system, and the emergent themes indicate that there are ripe opportunities for future study.

Experience Report: Standards-Based Grading at Scale in Algorithms

How We Manage an Army of Teaching Assistants: Experience Report on Scaling a CS1 Course

A Berkeley View of Teaching CS at Scale

Grading-Based Test Suite Augmentation.

Teaching the art of functional programming using automated grading (experience report)

Enhancing Student Engagement in Large-Scale Capstone Courses: An Experience Report

Delving Into the Design and Implementation of Specifications Grading Systems in Higher Education: A Systematic Review

SimGrade: Using Code Similarity Measures for More Accurate Human Grading

An Analysis of Programming Course Evaluations Before and After the Introduction of an Autograder

Integrating Software Engineering Key Practices into an OOP Massive In-Classroom Course: an Experience Report

"It Can Relate to Real Lives": Attitudes and Expectations in Justice-Centered Data Structures & Algorithms for Non-Majors

Assessment of Programming Language Learning Based on Peer Code Review Model: Implementation and Experience Report

Where Does Experience Matter in Software Process Education? an Experience Report.

Automated assessment system for programming courses: a case study for teaching data structures and algorithms

Generative Grading: Near Human-level Accuracy for Automated Feedback on Richly Structured Problems

Formative Assessment of Programming Language Learning Based on Peer Code Review: Implementation and Experience Report

Design and Implementation of Machine Learning Algorithms in Automatic Grading of Students' Assignments

Code Interviews: Design and Evaluation of a More Authentic Assessment for Introductory Programming Assignments

Program Equivalence for Assisted Grading of Functional Programs (Extended Version)

PowerGrader: Automating Code Assessment Based on PowerShell for Programming Courses.

Surfacing Equity Issues in Large Computing Courses with Peer-Ranked, Demographically-Labeled Student Feedback