Abstract:Immersive software tools are virtual environments designed to give their users an augmented view of real-world data and ways of manipulating that data. As virtual environments, every action users make while interacting with these tools can be carefully logged, as can the state of the software and the information it presents to the user, giving these actions context. This data provides a high-resolution lens through which dynamic cognitive and behavioral processes can be viewed. In this report, we describe new methods for the analysis and interpretation of such data, utilizing a novel implementation of the Beta Process Hidden Markov Model (BP-HMM) for analysis of software activity logs. We further report the results of a preliminary study designed to establish the validity of our modeling approach. A group of 20 participants were asked to play a simple computer game, instrumented to log every interaction with the interface. Participants had no previous experience with the game's functionality or rules, so the activity logs collected during their naïve interactions capture patterns of exploratory behavior and skill acquisition as they attempted to learn the rules of the game. Pre- and post-task questionnaires probed for self-reported styles of problem solving, as well as task engagement, difficulty, and workload. We jointly modeled the activity log sequences collected from all participants using the BP-HMM approach, identifying a global library of activity patterns representative of the collective behavior of all the participants. Analyses show systematic relationships between both pre- and post-task questionnaires, self-reported approaches to analytic problem solving, and metrics extracted from the BP-HMM decomposition. Overall, we find that this novel approach to decomposing unstructured behavioral data within software environments provides a sensible means for understanding how users learn to integrate software functionality for strategic task pursuit.

Analyzing repetitive action in game based on sequence pattern matching

"And then they died": Using Action Sequences for Data Driven,Context Aware Gameplay Analysis

BPMiner: mining developers' behavior patterns from screen-captured task videos.

Using gameplay data to examine learning behavior patterns in a serious game

Visual Analysis of Gameplay Data

Investigating the effects of motion-based Kinect game system on user cognition

Analysis of Boardgames using Eye-tracking: Case Study with Gomoku

Capturing Sequences of Learners' Self-Regulatory Interactions With Instructional Material During Game-Based Learning Using Auto-Recurrence Quantification Analysis

Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences

Taking the Scenic Route: Automatic Exploration for Videogames

Sequence Analysis of Learning Behavior in Different Consecutive Activities

EXPLORING GAZE BEHAVIOR TO ASSESS PERFORMANCE IN DIGITAL GAME-BASED LEARNING SYSTEMS

Player Experience Extraction from Gameplay Video

A Somatosensory Interaction System based on Kinect

What We Can Learn from the Data: A Multiple-Case Study Examining Behavior Patterns by Students with Different Characteristics in Using a Serious Game

Modeling strategic use of human computer interfaces with novel hidden Markov models

Patterns in Serious Game Design and Evaluation Application of Eye-Tracker and Biosensors

Understanding Players' Interaction Patterns with Mobile Game App UI via Visualizations

Research and Application of Running Action Sequence Recognition Algorithms Based on Kinect

Building a Behavioral Profile and Assessing the Skill of Video Game Players

A Study on the Analysis on Manipulation System of MORPG Game based on Finger Movement and Key Input Data