Abstract:We present the results of an exploratory study on how pairs interact with speech commands and touch gestures on a wall-sized display during a collaborative sensemaking task. Previous work has shown that speech commands, alone or in combination with other input modalities, can support visual data exploration by individuals. However, it is still unknown whether and how speech commands can be used in collaboration, and for what tasks. To answer these questions, we developed a functioning prototype that we used as a technology probe. We conducted an in-depth exploratory study with 10 participant pairs to analyze their interaction choices, the interplay between the input modalities, and their collaboration. While touch was the most used modality, we found that participants preferred speech commands for global operations, used them for distant interaction, and that speech interaction contributed to the awareness of the partner's actions. Furthermore, the likelihood of using speech commands during collaboration was related to the personality trait of agreeableness. Regarding collaboration styles, participants interacted with speech equally often whether they were in loosely or closely coupled collaboration. While the partners stood closer to each other during close collaboration, they did not distance themselves to use speech commands. From our findings, we derive and contribute a set of design considerations for collaborative and multimodal interactive data analysis systems. All supplemental materials are available at <a class="link-external link-https" href="https://osf.io/8gpv2" rel="external noopener nofollow">this https URL</a>

Speech + Posture: A Method for Interaction with Multiple and Large Interactive Displays

COMTIS: Customizable Touchless Interaction System for Large Screen Visualization

Promoted Motion-Sensing Gestural Interaction by Adapting to Group Users' Spatial Movements

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Interaction preferences of human body's natural gestures and gestures in smart Large displays scenarios

A large display-based approach supporting natural user interaction in virtual reality environment

Design of a speech interaction system for man-machine confrontation

Using the CAT for 3D Sketching in Front of Large Displays

Facilitating Interaction with Large Displays in Smart Spaces

Design and Development of a Gesture-Based Interaction System for Multi-Projector Tiled Display Wall

Interaction on Large Display Using Structured Laser Pointer

A Solution of Human-Computer Remote Interaction with Tactile Feedback

Asymmetric Free-Hand Interaction on a Large Display and Inspirations for Designing Natural User Interfaces

Design And Evaluation Of Cross-Objects User Interface For Whiteboard Interaction

42.4L: Late‐News Paper: Footprint of Scalable 3D Telecommunication: Using Integral Light Field Display and Kinect‐based Capture

Interaction With Gaze, Gesture, and Speech in a Flexibly Configurable Augmented Reality System

LADICA: A Large Shared Display Interface for Generative AI Cognitive Assistance in Co-Located Team Collaboration

Talk to the Wall: The Role of Speech Interaction in Collaborative Visual Analytics

A multimodal interface to situated assembly robot systems

A Tangible Multi-Display Toolkit to Support the Collaborative Design Exploration of AV-Pedestrian Interfaces

Magic music desk: a multi-modal embodied interactive desk