Abstract:The increasing use of digital learning tools and platforms in formal and informal learning settings has provided broad access to large amounts of learner data, the analysis of which has been aimed at understanding students' learning processes, improving learning outcomes, providing learner support as well as teaching. Presently, such data has been largely accessed from discussion forums in online learning management systems and has been further analyzed through the application of social network analysis (SNA). Nevertheless, the results of these analyses have not always been reproducible. Since such learning analytics (LA) methods rely on measurement as a first step of the process, the robustness of selected techniques for measuring collaborative learning activities is critical for the transparency, reproducibility and generalizability of the results. This paper presents findings from a study focusing on the validation of critical centrality measures frequently used in the fields of LA and SNA research. We examined how different network configurations (i.e., multigraph, weighted, and simplified) influence the reproducibility and robustness of centrality measures as indicators of student learning in CSCL settings. In particular, this research aims to contribute to the provision of robust and valid methods for measuring and better understanding of the participation and social dimensions of collaborative learning. The study was conducted based on a dataset of 12 university courses. The results show that multigraph configuration produces the most consistent and robust centrality measures. The findings also show that degree centralities calculated with the multigraph methods are reliable indicators for students' participatory efforts as well as a consistent predictor of their performance. Similarly, Eigenvector centrality was the most consistent centrality that reliably represented social dimension, regardless of the network configuration. This study offers guidance on the appropriate network representation as well as sound recommendations about how to reliably select the appropriate metrics for each dimension.

Reviewing Mixed Methods Approaches Using Social Network Analysis for Learning and Education

Linking Quantitative and Qualitative Network Approaches: A Review of Mixed Methods Social Network Analysis in Education Research

Mixed methods with social network analysis for networked learning

Combining social network analysis and agent-based modelling to explore dynamics of human interaction: A review

A Survey on Studying the Social Networks of Students

Social Network Analysis in Human Resource Development: A New Methodology

Macro, meso, micro: broadening the 'social' of social network analysis with a mixed methods approach

Leveraging social network analysis in education through social robots: A review

Social networks and the study of relations: networks as method, metaphor and form

Mixed methods research in library and information science: A methodological review

Planning and Conducting Mixed Methods Studies in Mathematics Educational Research

Mixed Methods for Research on Support Networks of People Experiencing Chronic Illness and Social Marginalization

Social network analysis and public policy: what’s new?

Social Network Analysis: A Survey on Process, Tools, and Application

Use of social network analysis in health research: a scoping review protocol

Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter?

The Development of Social Network Analysis Research in Mainland China: a Literature Review Perspective.

From Social Networks to Publishing Platforms: A Review of the History and Scholarship of Academic Social Network Sites

A Review of Latent Space Models for Social Networks

An Ontology-Based multi-domain model in Social Network Analysis: Experimental validation and case study

Social Network Analysis: An Example of Fusion Between Quantitative and Qualitative Methods