Abstract:Writing essays is an important skill which enables one to clearly write the ideas and understanding of certain topic with the help of language articulation and examples. Writing essay is a skill so is the grading of those essays. It requires a lot of efforts to grade these essays and the task becomes tedious and repetitive when the student to teacher ratio is high. As with any other repetitive task, the intervention of technology for automated essay grading has been thought of long back. However, the main challenge in automated essay grading lies in the understanding of language construction, word usage and presentation of idea/ argument/ narration. Language complexity makes natural language understanding a challenging task. In this work, we show our experiments with pre-trained static word embeddings like GloVe, fastText and pre-trained contextual model Bidirectional Encoder Representations from Transformers (BERT) for the task of automated essay grading. For the regression task, we have used Long Short-Term Memory (LSTM) and Support Vector Regression (SVR) models under various feature settings framed from the learnt embeddings. The results are shown with the ASAP-AES dataset on all 8 prompts. Our work shows average Quadratic Weighted Kappa (QWK) of 0.81 and 0.71 with SVR and LSTM on in-domain test set essays, respectively. The SVR model shows a better QWK than the human-human agreement of 0.75. To the best of our knowledge, our SVR model with pre-trained BERT embeddings achieve the highest average QWK reported on ASAP-AES data set. We further show the performance of our approach with adversary samples generated using permuted essays and off-topic essays. We experimentally show that our LSTM model though does not show high QWK score with human assigned grade but is robust against the adversarial settings considered.

Understanding Expert Knowledge for Chinese Essay Grading

An Automated Grader for Chinese Essay Combining Shallow and Deep Semantic Attributes

A Comparative Study of Pre-Service Teachers’ Views on Ethical Issues in Classroom Assessment in China and the United States

Computer-Aided English Education in China: An Online Automatic Essay Scoring System

Human-AI Collaborative Essay Scoring: A Dual-Process Framework with LLMs

Examining the Effects of a Real-Time, Knowledge-Aware Tool for Academic Writing Assessment

Grade Like a Human: Rethinking Automated Assessment with Large Language Models

Modeling essay grading with pre-trained BERT features

Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring.

Research on the optimization of essay scores based on deep learning

Grading the Graders: Comparing Generative AI and Human Assessment in Essay Evaluation

Automated Essay Scoring Using Grammatical Variety and Errors with Multi-Task Learning and Item Response Theory

Comment Text Grading for Chinese Graduate Academic Dissertation Using Attention Convolutional Neural Networks

Hey AI Can You Grade My Essay?: Automatic Essay Grading

Scoring method of English composition integrating deep learning in higher vocational colleges

Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring

A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimization

The impact of essay organization and overall quality on the holistic scoring of EFL writing: Perspectives from classroom english teachers and national writing raters

Design and Implementation of Machine Learning Algorithms in Automatic Grading of Students' Assignments

Automatic Short Answer Grading via Multiway Attention Networks

Automatic Essay Multi-dimensional Scoring with Fine-tuning and Multiple Regression