Abstract:Automated essay scoring (AES) is a compelling topic in Learning Analytics for the primary reason that recent advances in AI find it as a good testbed to explore artificial supplementation of human creativity. However, a vast swath of research tackles AES only holistically; few have even developed AES models at the rubric level, the very first layer of explanation underlying the prediction of holistic scores. Consequently, the AES black box has remained impenetrable. Although several algorithms from Explainable Artificial Intelligence have recently been published, no research has yet investigated the role that these explanation models can play in: (a) discovering the decision-making process that drives AES, (b) fine-tuning predictive models to improve generalizability and interpretability, and (c) providing personalized, formative, and fine-grained feedback to students during the writing process. Building on previous studies where models were trained to predict both the holistic and rubric scores of essays, using the Automated Student Assessment Prize’s essay datasets, this study focuses on predicting the quality of the writing style of Grade-7 essays and exposes the decision processes that lead to these predictions. In doing so, it evaluates the impact of deep learning (multi-layer perceptron neural networks) on the performance of AES. It has been found that the effect of deep learning can be best viewed when assessing the trustworthiness of explanation models. As more hidden layers were added to the neural network, the descriptive accuracy increased by about 10%. This study shows that faster (up to three orders of magnitude) SHAP implementations are as accurate as the slower model-agnostic one. It leverages the state-of-the-art in natural language processing, applying feature selection on a pool of 1592 linguistic indices that measure aspects of text cohesion, lexical diversity, lexical sophistication, and syntactic sophistication and complexity. In addition to the list of most globally important features, this study reports (a) a list of features that are important for a specific essay (locally), (b) a range of values for each feature that contribute to higher or lower rubric scores, and (c) a model that allows to quantify the impact of the implementation of formative feedback.

Research on the optimization of essay scores based on deep learning

Comparative Study about the Impacts of Chinese Automated Writing Evaluation (AWE) Feedback and Teacher Feedback on Middle School Students’ Writing Practice

An Automated Grader for Chinese Essay Combining Shallow and Deep Semantic Attributes

Scoring method of English composition integrating deep learning in higher vocational colleges

Machine Learning-Based Intelligent Scoring System for English Essays under the Background of Modern Information Technology

Research and Design of Automatic Scoring Algorithm for English Composition Based on Machine Learning

An intelligent scoring system for English writing based on artificial intelligence and machine learning

Machine learning approach of Japanese composition scoring and writing aided system's design

Interactive intelligent teaching and automatic composition scoring system based on linear regression machine learning algorithm

Automated Essay Feedback Generation and Its Impact on Revision

ELion: An Intelligent Chinese Composition Tutoring System Based on Large Language Models

College English Intelligent Writing Score System Based on Big Data Analysis and Deep Learning Algorithm

Computer-Aided English Education in China: An Online Automatic Essay Scoring System

The Impact of Automated Evaluation Feedback on Students’ Writing Revision

Analyse Chinese Teaching Project-Based Learning Design Application Optimization Using Keras Model

Intelligent Scoring of English Composition by Machine Learning from the Perspective of Natural Language Processing

Automatic Essay Multi-dimensional Scoring with Fine-tuning and Multiple Regression

A Model-based Feature Optimization Approach to Chinese Language Processing

Explainable Automated Essay Scoring: Deep Learning Really Has Pedagogical Value

Research on Automatic Error Correction Method in English Writing Based on Deep Neural Network

Questioning Optimization Strategies of High School English Reading Classroom based on Deep Learning