Automatic Essay Multi-dimensional Scoring with Fine-tuning and Multiple Regression

Kun Sun,Rong Wang

2024-06-03

Abstract:Automated essay scoring (AES) involves predicting a score that reflects the writing quality of an essay. Most existing AES systems produce only a single overall score. However, users and L2 learners expect scores across different dimensions (e.g., vocabulary, grammar, coherence) for English essays in real-world applications. To address this need, we have developed two models that automatically score English essays across multiple dimensions by employing fine-tuning and other strategies on two large datasets. The results demonstrate that our systems achieve impressive performance in evaluation using three criteria: precision, F1 score, and Quadratic Weighted Kappa. Furthermore, our system outperforms existing methods in overall scoring.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the issue of achieving multidimensional scoring in Automated Essay Scoring (AES). Traditional AES systems typically provide only an overall score, while users and second language learners expect to receive scores across multiple dimensions (such as vocabulary, grammar, coherence, etc.). To meet this demand, the authors developed two models that implement multidimensional automated scoring of English essays on two large datasets through fine-tuning and various strategies. Experimental results show that these systems perform excellently in terms of accuracy, F1 score, and Quadratic Weighted Kappa (QWK) across three evaluation criteria, and they outperform existing methods in overall scoring.