Abstract:Abstract Process data are becoming more and more popular in education research. In the field of computer-based assessments of collaborative problem solving (ColPS), process data have been used to identify students’ test-taking strategies while working on the assessment, and such data can be used to complement data collected on accuracy and overall performance. Such information can be used to understand, for example, whether students are able to use a range of styles and strategies to solve different problems, given evidence that such cognitive flexibility may be important in labor markets and societies. In addition, process information might help researchers better identify the determinants of poor performance and interventions that can help students succeed. However, this line of research, particularly research that uses these data to profile students, is still in its infancy and has mostly been centered on small- to medium-scale collaboration settings between people (i.e., the human-to-human approach). There are only a few studies involving large-scale assessments of ColPS between a respondent and computer agents (i.e., the human-to-agent approach), where problem spaces are more standardized and fewer biases and confounds exist. In this study, we investigated students’ ColPS behavioral patterns using latent profile analyses (LPA) based on two types of process data (i.e., response times and the number of actions) collected from the Program for International Student Assessment (PISA) 2015 ColPS assessment, a large-scale international assessment of the human-to-agent approach. Analyses were conducted on test-takers who: (a) were administered the assessment in English and (b) were assigned the Xandar unit at the beginning of the test. The total sample size was N = 2,520. Analyses revealed two profiles (i.e., Profile 1 [95%] vs. Profile 2 [5%]) showing different behavioral characteristics across the four parts of the assessment unit. Significant differences were also found in overall performance between the profiles.

Latent Class Analysis of Recurrent Events in Problem-Solving Items

Statistical Analysis of Complex Problem-Solving Process Data: an Event History Analysis Approach

DIAGNOSTIC Classification Analysis of Problem-Solving Competence using Process Data: An Item Expansion Method

Biclustering of Log Data: Insights from a Computer-Based Complex Problem Solving Assessment

Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015

A Continuous-Time Dynamic Choice Measurement Model for Problem-Solving Process Data

Latent Feature Extraction for Process Data Via Multidimensional Scaling

External Correlates of Adult Digital Problem-Solving Process

Logfile analyses of successful and unsuccessful strategy use in complex problem-solving: a cross-national comparison study

Understanding students’ problem-solving patterns: Evidence from an allotted response time in a PISA 2012 item

Analysis of Log Data from an International Online Educational Assessment System: A Multi-state Survival Modeling Approach to Reaction Time between and across Action Sequence

An Exploratory Analysis of the Latent Structure of Process Data Via Action Sequence Autoencoders

External Correlates of Adult Digital Problem-Solving Behavior: Log Data Analysis of a Large-Scale Assessment

Exploring Latent Constructs through Multimodal Data Analysis

Learning analysis of K-12 students’ online problem solving: a three-stage assessment approach

A Latent Hidden Markov Model for Process Data

Subtask analysis of process data through a predictive model

Investigating response behavior through TF-IDF and Word2vec text analysis: A case study of PISA 2012 problem-solving process data

A Latent Topic Model with Markov Transition for Process Data

A Probabilistic Framework for Temporal Cognitive Diagnosis in Online Learning Systems

On Latent Structure Examination of Behavioral Measuring Instruments in Complex Empirical Settings