Abstract:We present a visual analytics approach for multi-level visual exploration of users' interaction strategies in an interactive digital environment. The use of interactive touchscreen exhibits in informal learning environments, such as museums and science centers, often incorporate frameworks that classify learning processes, such as Bloom's taxonomy, to achieve better user engagement and knowledge transfer. To analyze user behavior within these digital environments, interaction logs are recorded to capture diverse exploration strategies. However, analysis of such logs is challenging, especially in terms of coupling interactions and cognitive learning processes, and existing work within learning and educational contexts remains limited. To address these gaps, we develop a visual analytics approach for analyzing interaction logs that supports exploration at the individual user level and multi-user comparison. The approach utilizes algorithmic methods to identify similarities in users' interactions and reveal their exploration strategies. We motivate and illustrate our approach through an application scenario, using event sequences derived from interaction log data in an experimental study conducted with science center visitors from diverse backgrounds and demographics. The study involves 14 users completing tasks of increasing complexity, designed to stimulate different levels of cognitive learning processes. We implement our approach in an interactive visual analytics prototype system, named VISID, and together with domain experts, discover a set of task-solving exploration strategies, such as "cascading" and "nested-loop', which reflect different levels of learning processes from Bloom's taxonomy. Finally, we discuss the generalizability and scalability of the presented system and the need for further research with data acquired in the wild.

Student projects involving novel interaction with large displays

COMTIS: Customizable Touchless Interaction System for Large Screen Visualization

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Design and Development of a Gesture-Based Interaction System for Multi-Projector Tiled Display Wall

Exploitation Of Novel Multiplayer Gesture-Based Interaction And Virtual Puppetry For Digital Storytelling To Develop Children'S Narrative Skills

Real-time Interaction Platform for Classroom Teaching Based on Smart Phone APP

Going Beyond Second Screens: Applications for the Multi-display Intelligent Living Room

3D Intuitive Gesture Interaction via Motion Sensing

Interactive 3D Editing on Tiled Display Wall

Motion-Based Perceptual User Interface

A large display-based approach supporting natural user interaction in virtual reality environment

Design And Evaluation Of Cross-Objects User Interface For Whiteboard Interaction

Digital Interaction with Physical Museum Artifacts

Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

Design of an interactive system of tangible interfaces to support learning in children with autism

Smart Campus: Fostering the Community Awareness Through an Intelligent Environment

Design of a new human-computer interactive device for projection display

Revealing Interaction Dynamics: Multi-Level Visual Exploration of User Strategies with an Interactive Digital Environment

An Interactive System for Exhibitions in a Science and Technology Center

Vision-Based Projected Tabletop Interface For Finger Interactions

Facilitating Interaction with Large Displays in Smart Spaces