Abstract:Immersive software tools are virtual environments designed to give their users an augmented view of real-world data and ways of manipulating that data. As virtual environments, every action users make while interacting with these tools can be carefully logged, as can the state of the software and the information it presents to the user, giving these actions context. This data provides a high-resolution lens through which dynamic cognitive and behavioral processes can be viewed. In this report, we describe new methods for the analysis and interpretation of such data, utilizing a novel implementation of the Beta Process Hidden Markov Model (BP-HMM) for analysis of software activity logs. We further report the results of a preliminary study designed to establish the validity of our modeling approach. A group of 20 participants were asked to play a simple computer game, instrumented to log every interaction with the interface. Participants had no previous experience with the game's functionality or rules, so the activity logs collected during their naïve interactions capture patterns of exploratory behavior and skill acquisition as they attempted to learn the rules of the game. Pre- and post-task questionnaires probed for self-reported styles of problem solving, as well as task engagement, difficulty, and workload. We jointly modeled the activity log sequences collected from all participants using the BP-HMM approach, identifying a global library of activity patterns representative of the collective behavior of all the participants. Analyses show systematic relationships between both pre- and post-task questionnaires, self-reported approaches to analytic problem solving, and metrics extracted from the BP-HMM decomposition. Overall, we find that this novel approach to decomposing unstructured behavioral data within software environments provides a sensible means for understanding how users learn to integrate software functionality for strategic task pursuit.

Using lag-sequential analysis for understanding interaction sequences in visualizations

Revealing Interaction Dynamics: Multi-Level Visual Exploration of User Strategies with an Interactive Digital Environment

Lag sequential analysis: Robust statistical methods.

Sequence Modelling For Analysing Student Interaction with Educational Systems

Finding a Clear Path: Structuring Strategies for Visualization Sequences

Lag Sequential Analysis: Taking Consultation Communication Research Methodology to the Movies

What User Behaviors Make the Differences During the Process of Visual Analytics?

Visualizing Event Sequence Data for User Behavior Evaluation of In-Vehicle Information Systems

Analyzing repetitive action in game based on sequence pattern matching

Modeling strategic use of human computer interfaces with novel hidden Markov models

QLens: Visual Analytics of Multi-step Problem-solving Behaviors for Improving Question Design

Deducing self-interaction in eye movement data using sequential spatial point processes

Recording Reusable and Guided Analytics From Interaction Histories

"And then they died": Using Action Sequences for Data Driven,Context Aware Gameplay Analysis

Understanding social welfare service patterns using sequential analysis

Analyzing Multimodal Interaction Strategies for LLM-Assisted Manipulation of 3D Scenes

Survey on Visual Analysis of Event Sequence Data

Examining Interaction Techniques in Data Visualization Authoring Tools from the Perspective of Goals and Human Cognition: a Survey

The Sequence Matters in Learning -- A Systematic Literature Review

Talk to the Wall: The Role of Speech Interaction in Collaborative Visual Analytics

Do You See What I See? A Qualitative Study Eliciting High-Level Visualization Comprehension