Abstract:We propose and study a novel cross-reality environment that seamlessly integrates a monoscopic 2D surface (an interactive screen with touch and pen input) with a stereoscopic 3D space (an augmented reality HMD) to jointly host spatial data visualizations. This innovative approach combines the best of two conventional methods of displaying and manipulating spatial 3D data, enabling users to fluidly explore diverse visual forms using tailored interaction techniques. Providing such effective 3D data exploration techniques is pivotal for conveying its intricate spatial structures -- often at multiple spatial or semantic scales -- across various application domains and requiring diverse visual representations for effective visualization. To understand user reactions to our new environment, we began with an elicitation user study, in which we captured their responses and interactions. We observed that users adapted their interaction approaches based on perceived visual representations, with natural transitions in spatial awareness and actions while navigating across the physical surface. Our findings then informed the development of a design space for spatial data exploration in cross-reality. We thus developed cross-reality environments tailored to three distinct domains: for 3D molecular structure data, for 3D point cloud data, and for 3D anatomical data. In particular, we designed interaction techniques that account for the inherent features of interactions in both spaces, facilitating various forms of interaction, including mid-air gestures, touch interactions, pen interactions, and combinations thereof, to enhance the users' sense of presence and engagement. We assessed the usability of our environment with biologists, focusing on its use for domain research. In addition, we evaluated our interaction transition designs with virtual and mixed-reality experts to gather further insights.

Evaluating Situated Visualization in AR with Eye Tracking

What We See and What We Get from Visualization: Eye Tracking Beyond Gaze Distributions and Scanpaths

Evaluation of indoor guidance systems using eye tracking in an immersive virtual environment

Visualization Psychology for Eye Tracking Evaluation

The Eye in Extended Reality: A Survey on Gaze Interaction and Eye Tracking in Head-worn Extended Reality

Assessing Augmented Reality Selection Techniques for Passengers in Moving Vehicles: A Real-World User Study

Usage and Effect of Eye Tracking in Remote Guidance

Design Patterns for Situated Visualization in Augmented Reality

Towards an Understanding of Situated AR Visualization for Basketball Free-Throw Training

Interactive Multi-User 3D Visual Analytics in Augmented Reality

Survey and Evaluation of Monocular Visual-Inertial SLAM Algorithms for Augmented Reality

Visual Analytics Methodology for Eye Movement Studies

Spatialgaze: Towards Spatial Gaze Tracking for Extended Reality

Eye-Tracking in Immersive Virtual Reality for Education: A Review of the Current Progress and Applications

Assessment of Human Behavior in Virtual Reality by Eye Tracking

Eye Tracking on Text Reading with Visual Enhancements

Towards Everyday Virtual Reality through Eye Tracking

Eye-tracking on virtual reality: a survey

SpatialTouch: Exploring Spatial Data Visualizations in Cross-reality

See or Hear? Exploring the Effect of Visual and Audio Hints and Gaze-assisted Task Feedback for Visual Search Tasks in Augmented Reality

Quantifying Dwell Time With Location-based Augmented Reality: Dynamic AOI Analysis on Mobile Eye Tracking Data With Vision Transformer