Abstract:With the rapid development of human–computer interaction technologies, the new media generation demands novel learning experiences with natural interaction and immersive experience. Considering that digital storytelling is a powerful pedagogical tool for young children, in this paper, we design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head‐mounted displays and depth motion sensor. In this system, the young players could intuitively use hand gestures to manipulate virtual puppets to perform a story and interact with props in a virtual stereoscopic environment. We have conducted a user experiment with four young children for pedagogical evaluation, as well as system acceptability and interactivity evaluation by postgraduate students. The results show that our framework has great potential to stimulate learning abilities of young children through collaboration tasks. The stereoscopic head‐mounted display outperformed the traditional monoscopic display in a comparison between the two.

Multi-channel Technology Based Animated Interaction in E-paper Device

A Comparative Study of Map Exploration Interfaces for Multi-Touch Tabletops.

A Multitouch Drawing Application With Occlusion-Free Interaction Strategies

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

A novel interactive exhibition method based on handheld devices

COMTIS: Customizable Touchless Interaction System for Large Screen Visualization

Multi-user collaborative exploration of map and decision-making on a multi-touch tabletop

Usability Impact of Occlusion-free Techniques on Commonly-used Multitouch Actions

Exploitation Of Novel Multiplayer Gesture-Based Interaction And Virtual Puppetry For Digital Storytelling To Develop Children'S Narrative Skills

Touch-Driven Interaction between Physical Space and Cyberspace with NFC.

A Penetrable Interactive 3D Display Based on Motion Recognition （invited Paper）

Hold, Touch and Read It: Border Interactions in Mobile Reading Environment

Design And Evaluation Of Cross-Objects User Interface For Whiteboard Interaction

Design of an Interactive Device Based on e-Textile Material

Touch-driven Interaction Via an NFC-enabled Smartphone

Motion-Based Perceptual User Interface

Tilt & Touch: Mobile Phone For 3d Interaction

Invited - Airtouch: a novel single layer 3D touch sensing system for human/mobile devices interactions

Promoted Motion-Sensing Gestural Interaction by Adapting to Group Users' Spatial Movements

Towards Video Display on Electronic Papers

An Interactive Hybrid Book Integrating Capacitive, Piezoelectric, and Piezoresistive Polymer‐Based Technologies