Game-Based Assessment of Students' Digital Literacy Using Evidence-Centered Game Design

Jiayuan Li,Jie Bai,Sha Zhu,Harrison Hao Yang
DOI: https://doi.org/10.3390/electronics13020385
IF: 2.9
2024-01-18
Electronics
Abstract:This study measured secondary students' digital literacy using a digital game-based assessment system that was designed and developed based on the Evidence-Centered Game Design (ECGD) approach. A total of 188 secondary students constituted the valid cases in this study. Fine-grained behavioral data generated from students' gameplay processes were collected and recorded with the assessment system. The Delphi method was used to extract feature variables related to digital literacy from the process data, and the Analytic Hierarchy Process (AHP) method was used to construct the measurement model. The assessment results of the ECGD-based assessment had a high correlation with standardized test scores, which have been shown to be reliable and valid in prior large-scale assessment studies.
engineering, electrical & electronic,computer science, information systems,physics, applied
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to address the limitations in current methods of assessing students' digital literacy, particularly in terms of evaluation content and tools. Specifically: 1. **Limitations of existing assessment methods**: - **Evaluation content**: Current evaluation content mainly focuses on the traditional "question-answer" model, emphasizing students' basic cognitive abilities such as digital knowledge and application skills. It fails to adequately assess higher-order cognitive abilities such as problem-solving using digital technology, innovation, and addressing ethical issues in the digital society. - **Evaluation tools**: Existing evaluation tools primarily rely on standardized tests and self-assessment scales. While these methods can provide an overall evaluation of students' digital literacy, they are not suitable for measuring implicit cognitive abilities and thought processes. 2. **Proposed new method**: - The paper proposes an Evidence-Centered Game Design (ECGD) approach to assess students' digital literacy. By developing a game-based assessment tool, fine-grained process data is collected as students complete tasks. The Delphi method and Analytic Hierarchy Process (AHP) are used to determine the extracted feature variables and their weights. - Through this method, the paper aims to provide a more reliable and effective assessment method to accurately measure students' digital literacy levels and validate its effectiveness. Through this research, the paper aims to promote the iterative optimization of student digital literacy assessment methods, better adapting to the needs of future education and social development.