Abstract:The increasing use of digital learning tools and platforms in formal and informal learning settings has provided broad access to large amounts of learner data, the analysis of which has been aimed at understanding students' learning processes, improving learning outcomes, providing learner support as well as teaching. Presently, such data has been largely accessed from discussion forums in online learning management systems and has been further analyzed through the application of social network analysis (SNA). Nevertheless, the results of these analyses have not always been reproducible. Since such learning analytics (LA) methods rely on measurement as a first step of the process, the robustness of selected techniques for measuring collaborative learning activities is critical for the transparency, reproducibility and generalizability of the results. This paper presents findings from a study focusing on the validation of critical centrality measures frequently used in the fields of LA and SNA research. We examined how different network configurations (i.e., multigraph, weighted, and simplified) influence the reproducibility and robustness of centrality measures as indicators of student learning in CSCL settings. In particular, this research aims to contribute to the provision of robust and valid methods for measuring and better understanding of the participation and social dimensions of collaborative learning. The study was conducted based on a dataset of 12 university courses. The results show that multigraph configuration produces the most consistent and robust centrality measures. The findings also show that degree centralities calculated with the multigraph methods are reliable indicators for students' participatory efforts as well as a consistent predictor of their performance. Similarly, Eigenvector centrality was the most consistent centrality that reliably represented social dimension, regardless of the network configuration. This study offers guidance on the appropriate network representation as well as sound recommendations about how to reliably select the appropriate metrics for each dimension.

A Learnable Content & Participation Analysis Toolkit for Assessing CSCL Learning Outcomes and Processes

Assessing Collaborative Process in CSCL with an Intelligent Content Analysis Toolkit

CSCL Interaction Analysis for Assessing Knowledge Building Outcomes: Method and Tool

A Tool for Analyzing Interaction in CSCL and a Case Study

The Three-Iterative Design and Implementation of a Student-Facing Social Learning Analytics Tool

Analyzing Peer Interactions in Computer-Supported Collaborative Learning: Model, Method and Tool

Computer-Supported Content Analysis for Collaborative Knowledge Building in CSCL

Learning Analytics for Computer-Supported Collaborative Learning Design

Understanding Learners' Interaction Patterns Through Analyzing Context of Questions in Discussion Transcripts

ASSESSMENT INTELLIGENCE IN SMALL GROUP LEARNING

A Learning Analytics System for Cognition Analysis in Online Learning Community.

Research on the Function of CSCL Learning Analytics Tools based on Group Awareness

Analysis Of Socially Shared Regulation In Cscl

A systemic model of interaction analysis in CSCL

Classifying discourse in a CSCL platform to evaluate correlations with Teacher Participation and Progress

Analysis of Co-regulation Behavioral Patterns by Cluster and Sequential Analysis in CSCL

An automated group learning engagement analysis and feedback approach to promoting collaborative knowledge building, group performance, and socially shared regulation in CSCL

Interaction Pattern Analysis in Cmoocs Based on the Connectivist Interaction and Engagement Framework

Using Learning Analytics to Support Engagement in Collaborative Writing

The Contents of an Online Discussion Forum: An Analysis

Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter?