Abstract:Automated scoring of subjective assignments (ASSA), one of the most representative applications in the field of artificial intelligence in education, is the process of scoring and assessing essays with the aid of natural language processing technology. Nevertheless, there are a few critical opinions regarding ASSA models. For instance, when essays of high or low quality are fed to an ASSA model, it may result in a great difference between the scores given by the model and human evaluators. To overcome these problems, this study is devoted to the exploration on human–machine collaborative scoring of subjective assignments based on sequential three-way decision. Firstly, we propose a human–machine task allocation model based on sequential three-way decisions (TA-S3WD) to establish a formal framework for human–machine collaborative scoring of subjective assignments. Secondly, we develop a human–machine collaborative subjective assignment scoring model by combining bidirectional long short-term memory (Bi-LSTM) with TA-S3WD model, called HMCS-BLTS. Finally, numerical experiments with intensive comparisons and sensitivity analysis are conducted on the automated student assessment prize (ASAP) data set to show the advantages of the proposed model. Comparative analysis shows that the HMCS-BLTS model achieves higher execution efficiency than seven well-known baseline models in terms of four common evaluation metrics. Moreover, compared with the best-performing baseline model, the HMCS-BLTS model obtains 0.8967 average quadratic weighted kappa and achieves 19.31% improvements on average using just 19.02% of the human workload.

A competition-based student assignment management system

Optimized Matching Between Students of the Automation Major and Their Specialized Project Design Assignment

A System for Automatic Assessment and Plagiarism Detection of Student Programs

Peer-Formulated Assignment Method for Experimental Projects in CS Courses

A Competition-Oriented Student Team Building Method

Ant Colony Optimization Based Simulation Analysis of Sports Course Achievement Management System

Adaptive Curriculum Sequencing and Education Management System via Group-Theoretic Particle Swarm Optimization

A computer based study management system 1

Design and implementation of discipline competition management system

Using Assignment Incentives to Reduce Student Procrastination and Encourage Code Review Interactions

Motivating Students Through Peer-Formulated Assignments in CS Experimental Courses

Homework assisted assessment model study based on LMS 3 .0

Research on a VSM-based E-Homework Anti-Plagiarism System

Motivating Students with New Mechanisms of Online Assignments and Examination to Meet the MOOC Challenges for Programming

The Design and Development of A Semi-Auto Computer Generated Testing Paper System - A Case Study in the School of Continuing Education at China University of Geosciences

Automated Student Code Scoring by Analyzing Grammatical and Semantic Information of Code

Automated Assignment Management System: Integrating Email Inbox Parsing, Database Updates, and Notification

Assessing College Students Writing Skill: A Computer Aided Assessment System (CAAS)

Human-machine Collaborative Scoring of Subjective Assignments Based on Sequential Three-Way Decisions

Reviewer Assignment Strategy of Peer Assessment: Towards Managing Collusion in Self-assignment

Design and Implementation of Machine Learning Algorithms in Automatic Grading of Students' Assignments