Measuring the Effectiveness of Learning Resources Via Student Interaction with Online Learning Modules

Zhongzhou Chen,Matthew Guthrie

DOI: https://doi.org/10.48550/arXiv.1903.08003

2019-03-19

Abstract:We present a new method for measuring the effectiveness of online learning resources, through the analysis of time-stamped log data of students' interaction with a sequence of online learning modules created based on the concept of mastery learning. Each module was designed to assess students' mastery of one topic before and after interacting with the learning resources in the module. In addition, analysis of log data provides information on students' test-taking effort when completing the assessment, and learning effort when interacting with the learning resources. Combining these three measurements provides accurate information on the quality of each online learning module, as well as detailed suggestions for future improvements. The results from data collected from all 10 modules are presented in a sequence of sunburst charts, an intuitive visual representation designed to allow the average instructor to quickly grasp the key outcomes of the data analysis, and identify less effective modules for future improvements. Online learning modules can be implemented much more frequently than either clinical experiments or classroom assessments, and can provide more interpretable data than most existing online courses, making it a valuable tool for quickly assessing the effectiveness of online learning resources.

Physics Education

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to effectively evaluate the extent to which online learning resources help students learn specific topics. Specifically, when a student group is assigned one or a set of learning resources, what proportion of students can these resources help improve their understanding or skills, and to what extent. Traditional methods such as clinical experiments and classroom evaluations have limitations, for example, they are difficult to implement, occur at a low frequency, and are difficult to directly relate to the quality of specific teaching resources. Therefore, this paper introduces a new method. By analyzing the timestamp log data of students' interactions with online learning modules designed based on the concept of mastery learning, it can quickly provide detailed information about the effectiveness of online learning resources. This method combines the advantages of existing methods and overcomes some major disadvantages, such as higher evaluation frequency, stronger data interpretability, and the ability to reflect students' real - learning environments.