Abstract:Text-based open-ended questions in academic formative and summative assessments help students become deep learners and prepare them to understand concepts for a subsequent conceptual assessment. However, grading text-based questions, especially in large (>50 enrolled students) courses, is tedious and time-consuming for instructors. Text processing models continue progressing with the rapid development of Artificial Intelligence (AI) tools and Natural Language Processing (NLP) algorithms. Especially after breakthroughs in Large Language Models (LLM), there is immense potential to automate rapid assessment and feedback of text-based responses in education. This systematic review adopts a scientific and reproducible literature search strategy based on the PRISMA process using explicit inclusion and exclusion criteria to study text-based automatic assessment systems in post-secondary education, screening 838 papers and synthesizing 93 studies. To understand how text-based automatic assessment systems have been developed and applied in education in recent years, three research questions are considered: 1) What types of automated assessment systems can be identified using input, output, and processing framework? 2) What are the educational focus and research motivations of studies with automated assessment systems? 3) What are the reported research outcomes in automated assessment systems and the next steps for educational applications? All included studies are summarized and categorized according to a proposed comprehensive framework, including the input and output of the system, research motivation, and research outcomes, aiming to answer the research questions accordingly. Additionally, the typical studies of automated assessment systems, research methods, and application domains in these studies are investigated and summarized. This systematic review provides an overview of recent educational applications of text-based assessment systems for understanding the latest AI/NLP developments assisting in text-based assessments in higher education. Findings will particularly benefit researchers and educators incorporating LLMs such as ChatGPT into their educational activities.

Semi-automatic Assessment Model of Student Texts - Pedagogical Foundations

Automatic short answer grading and feedback using text mining methods

Automatic assessment of text-based responses in post-secondary education: A systematic review

Machine Learning Mixed Methods Text Analysis: An Illustration From Automated Scoring Models of Student Writing in Biology Education

‘Finally studying for myself’ – examining student agency in summative and formative self-assessment models

Applying natural language processing to automatically assess student conceptual understanding from textual responses

Assessing digital multimodal composing in second language writing: Towards a process-based model

Beyond the grade: the planning, formative and summative (PFS) model of self-assessment for higher education

A pedagogical design for self-regulated learning in academic writing using text-based generative artificial intelligence tools: 6-P pedagogy of plan, prompt, preview, produce, peer-review, portfolio-tracking

Towards a Success Model for Automated Programming Assessment Systems Used as a Formative Assessment Tool

Examining the Effects of a Real-Time, Knowledge-Aware Tool for Academic Writing Assessment

An automated essay scoring systems: a systematic literature review

Automatic learner summary assessment for reading comprehension

Let's Ask Students About Their Programs, Automatically

Automated Essay Feedback Generation and Its Impact on Revision

Towards LLM-based Autograding for Short Textual Answers

Short Answer Grading Using One-shot Prompting and Text Similarity Scoring Model

Improving the quality of teaching by utilising written student feedback: A streamlined process

Distinguishing the Themes Emerging from Masses of Open Student Feedback

Construction of Intelligent Evaluation Model of English Composition Based on Machine Learning

Automated Writing Evaluation Using Sentence by Sentence Scoring Model