Abstract:Immersive software tools are virtual environments designed to give their users an augmented view of real-world data and ways of manipulating that data. As virtual environments, every action users make while interacting with these tools can be carefully logged, as can the state of the software and the information it presents to the user, giving these actions context. This data provides a high-resolution lens through which dynamic cognitive and behavioral processes can be viewed. In this report, we describe new methods for the analysis and interpretation of such data, utilizing a novel implementation of the Beta Process Hidden Markov Model (BP-HMM) for analysis of software activity logs. We further report the results of a preliminary study designed to establish the validity of our modeling approach. A group of 20 participants were asked to play a simple computer game, instrumented to log every interaction with the interface. Participants had no previous experience with the game's functionality or rules, so the activity logs collected during their naïve interactions capture patterns of exploratory behavior and skill acquisition as they attempted to learn the rules of the game. Pre- and post-task questionnaires probed for self-reported styles of problem solving, as well as task engagement, difficulty, and workload. We jointly modeled the activity log sequences collected from all participants using the BP-HMM approach, identifying a global library of activity patterns representative of the collective behavior of all the participants. Analyses show systematic relationships between both pre- and post-task questionnaires, self-reported approaches to analytic problem solving, and metrics extracted from the BP-HMM decomposition. Overall, we find that this novel approach to decomposing unstructured behavioral data within software environments provides a sensible means for understanding how users learn to integrate software functionality for strategic task pursuit.

Uncovering ecological state dynamics with hidden Markov models

Hidden Markov Models: Pitfalls and Opportunities in Ecology

An Introduction to Animal Movement Modeling with Hidden Markov Models using Stan for Bayesian Inference

Topological Hidden Markov Models

A hidden Markov movement model for rapidly identifying behavioral states from animal tracks

hmmTMB: Hidden Markov models with flexible covariate effects in R

Multi-scale modeling of animal movement and general behavior data using hidden Markov models with hierarchical structures

A Bayesian multilevel hidden Markov model with Poisson-lognormal emissions for intense longitudinal count data

Hidden Markov Model Variants and their Application

An Introduction to Hidden Markov Models

Signal Extraction from Long-Term Ecological Data Using Bayesian and Non-Bayesian State-Space Models

Comparing maximum likelihood and Bayesian methods for fitting hidden Markov models to multi-state capture-recapture data of invasive carp in the Illinois River

A guide to state–space modeling of ecological time series

Inference of genomic landscapes using ordered Hidden Markov Models with emission densities (oHMMed)

Variance-Reduced Stochastic Optimization for Efficient Inference of Hidden Markov Models

How to account for behavioral states in step-selection analysis: a model comparison

Improved state-space models for inference about spatial and temporal variation in abundance from count data

A Bayesian Hidden Semi-Markov Model with Covariate-Dependent State Duration Parameters for High-Frequency Environmental Data

Ecological non-linear state space model selection via adaptive particle Markov chain Monte Carlo (AdPMCMC)

Bayesian Nonparametric Hidden Semi-Markov Models

Modeling strategic use of human computer interfaces with novel hidden Markov models