Abstract:We propose and study a novel cross-reality environment that seamlessly integrates a monoscopic 2D surface (an interactive screen with touch and pen input) with a stereoscopic 3D space (an augmented reality HMD) to jointly host spatial data visualizations. This innovative approach combines the best of two conventional methods of displaying and manipulating spatial 3D data, enabling users to fluidly explore diverse visual forms using tailored interaction techniques. Providing such effective 3D data exploration techniques is pivotal for conveying its intricate spatial structures -- often at multiple spatial or semantic scales -- across various application domains and requiring diverse visual representations for effective visualization. To understand user reactions to our new environment, we began with an elicitation user study, in which we captured their responses and interactions. We observed that users adapted their interaction approaches based on perceived visual representations, with natural transitions in spatial awareness and actions while navigating across the physical surface. Our findings then informed the development of a design space for spatial data exploration in cross-reality. We thus developed cross-reality environments tailored to three distinct domains: for 3D molecular structure data, for 3D point cloud data, and for 3D anatomical data. In particular, we designed interaction techniques that account for the inherent features of interactions in both spaces, facilitating various forms of interaction, including mid-air gestures, touch interactions, pen interactions, and combinations thereof, to enhance the users' sense of presence and engagement. We assessed the usability of our environment with biologists, focusing on its use for domain research. In addition, we evaluated our interaction transition designs with virtual and mixed-reality experts to gather further insights.

Toward Spatial Temporal Consistency of Joint Visual Tactile Perception in VR Applications

Unveiling interactions of spatial-temporal information in tactile motion perception

A touchable virtual screen interaction system with handheld Kinect camera

Evaluating Visual-Spatiotemporal Co-Registration of a Physics-Based Virtual Reality Haptic Interface

Design, Optimization and Application of Tactile Sensor Based on Virtual Binocular Vision

Vibrotactile Representation of Three-Dimensional Shape and Implementation on A Vibrotactile Pad

Stretch or Vibrate? Rendering Spatial Information of Static and Moving Objects in VR via Haptic Feedback for Blind People

Model-Based 3D Contact Geometry Perception for Visual Tactile Sensor

SpatialTouch: Exploring Spatial Data Visualizations in Cross-reality

Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering

Virtual reality interaction based on visual attention and kinesthetic information

Haptic Perception, Mechanics, and Material Technologies for Virtual Reality

A Patterned Vibrotactile Method Using Envelope Modulation with High Resolution and Low Perceptual Frequency

A Proof of Concept Integrated Multi-systems Approach for Large Scale Tactile Feedback in VR

Data-Driven Spatio-Temporal Analysis Via Multi-Modal Zeitgebers and Cognitive Load in VR

Peripersonal Space Tele-Operation in Virtual Reality: The Role of Tactile - Force Feedback

Enhancing User Experience in Virtual Museums: Impact of Finger Vibrotactile Feedback

Haptic Sensing and Feedback Techniques toward Virtual Reality

Elastic Tactile Simulation Towards Tactile-Visual Perception.

Effect of temporality, physical activity and cognitive load on spatiotemporal vibrotactile pattern recognition