Abstract:Problem-solving competence is defined as the capacity to engage in cognitive processing to understand and resolve problem scenarios where a solution is not obvious. Computer-based assessments usually provide an interactive environment in which students can solve a problem by choosing among a set of available actions and taking one or more steps to complete a task. All students' actions are automatically recorded in system logs as coded and time-stamped strings. These strings are called process data. The process data have multi-level structures in which the actions are nested within a single individual and therefore they are logically interconnected. Recently, researches have focused on characterizing process data and analyzing the response strategies to solve the problem. This study proposed an extended MMixIRT model which incorporated the multilevel structure into a mixture IRT model. It can classify latent groups at process level that have different problem solving strategies, and estimate the students' abilities at the student level simultaneously. This model takes the accumulated response information as the specific steps at the process level and defines a more free matrix to determine the weight information used for ability estimation at the student level. Specifically, in the standard MMixIRT model, the student-level latent variables are generally obtained from the measurement results made by the process-level response variables, while students' final responses are used to estimate their problem-solving abilities in the extended MMixIRT model. This research applied process data recorded in one of the items (Traffic CP007Q02) of problem solving in PISA 2012. The samples were 3196 students from Canada, Hongkong-China, Shanghai-China, Singapore, and America. Based on the log file of the process record, there were 139,990 records in the final data file. It was found that (1) The model can capture different problem-solving strategies used by students at the process level, as well as provide ability estimates at the student level. (2) The model can also analyze the typical characteristics of students' strategy in problem-solving across different countries for targeted instructional interventions. It is concluded that the extended MMixIRT model can analyze response data at process and student levels. These analyses not only play an important role in the scoring, but also provide valuable information to psychometricians and test developers, help them to better understand what distinguishes well performing students from the ones that are not, and eventually lead to better test design.

Learning analysis of K-12 students’ online problem solving: a three-stage assessment approach

A Three-perspective Analysis to Study Problem Behaviors in High School Students

An Evaluation Of Elementary Students' Ability Of Problem Solving In Information Processing

UTILIZATION OF PROCESS DATA IN CHINA: EXPLORING STUDENTS' PROBLEM-SOLVING STRATEGIES IN COMPUTER- BASED SCIENCE ASSESSMENT FEATURING INTERACTIVE TASKS

An Eye Tracking Study of High- and Low-Performing Students in Solving Interactive and Analytical Problems.

The Analysis of the Influential Factors of Student’s Problem Solving Competence in Digital Test Environment-Taking PISA 2012 as an Example

Learning contextual factors, student engagement, and problem-solving skills: A Chinese perspective

Analysis of the Problem-solving strategies in computer-based dynamic assessment: The extension and application of multilevel mixture IRT model

Exploring the Relationship Between Students’ Information Problem Solving Patterns and Epistemic Beliefs: a Mixed Methods Sequential Analysis Study

The Analysis of Online Learning Behavior of the Students with Poor Academic Performance in Mathematics and Individual Help Strategies.

Can Students Identify the Relevant Information to Solve a Problem?

Statistical Analysis of Complex Problem-Solving Process Data: an Event History Analysis Approach

Can STEM Teaching Improve Students' Problem-Solving Ability: an Empirical Study in the Middle School

Exploring Patterns in Undergraduate Students' Information Problem Solving: A Cross-Case Comparison Study

Behavioral Patterns Associated with Solving Ill-Defined Complex Problems from a Multidimensional Perspective: Perception, Cognition, Metacognition, and Motivation

Developing assessment in improving students' digital literacy skills

Understanding students’ problem-solving patterns: Evidence from an allotted response time in a PISA 2012 item

Latent Class Analysis of Recurrent Events in Problem-Solving Items

Understanding Middle School Students’ Self-Efficacy and Performance in a Technology-Enriched Problem-Based Learning Program: A Learning Analytics Approach

What Is Problem-Solving Learning:Analysis Based on Three Theoretical Sources

Enhancing the Development of Chinese Fifth-Graders’ Problem-Posing and Problem-Solving Abilities, Beliefs, and Attitudes: A Design Experiment