A Quantitative Revision Method to Improve Usability of Self-and Peer Assessment in MOOCs

Jun Na,Yuan Liu

DOI: https://doi.org/10.1145/3321408.3322846

2019-01-01

Abstract:With the flourishing of Massive Open Online Courses (MOOCs), a course is no longer limited by any fix times and places. MOOCs enable a learner to access the contents of a world-wide course free from time and locations constrains, which lead to an unimaginable increasing amount of learners. Hence, it becomes a challenging problem in MOOCs that how to track all the students' learning progresses effectively and provide them timely feedbacks. To address this problem, self- and peer assessments have gained considerable momentum recently, as they can involve learners in their learning and the evaluation. However, as different students have different personalities, knowledge levels, senses of responsibility, and grading tactics, there are often many noisy data in the self- and peer grades that reduce their usability for giving an accurate and fairness final grade. In this paper, we study a self- and peer assessment dataset about a course with such an undesirable phenomenon that the grade inconsistency distributed over most of the students. To mitigate the observed problems, we propose a quantitative revision method by reducing the grading deviation gradually for all the students. We design an adjustment matrix to revise all the self- and peer grades in a global view and propose an algorithm to obtain the adjustment matrix based solely on the given noisy dataset. Moreover, we provide a method to measure the usability of a self- and peer grade dataset. Finally, we evaluate our approach in the collected self- and peer grading dataset. Experiment results show that our approach improves the usability of the obtained dataset at least 33%.