Abstract:With the rapid development of human–computer interaction technologies, the new media generation demands novel learning experiences with natural interaction and immersive experience. Considering that digital storytelling is a powerful pedagogical tool for young children, in this paper, we design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head‐mounted displays and depth motion sensor. In this system, the young players could intuitively use hand gestures to manipulate virtual puppets to perform a story and interact with props in a virtual stereoscopic environment. We have conducted a user experiment with four young children for pedagogical evaluation, as well as system acceptability and interactivity evaluation by postgraduate students. The results show that our framework has great potential to stimulate learning abilities of young children through collaboration tasks. The stereoscopic head‐mounted display outperformed the traditional monoscopic display in a comparison between the two.

Human-Computer Interaction Used in a Scene Simulation System Based on Vega

Realization of Chinese Characters Display Technology in Vega

Visual Simulation Design of Flight Based on Vega

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Development of Visual System of UAV by Vega

Development and Application of Integrated Human Machine Interaction Simulation System

An interaction system using mixed hand gestures.

2D and 3d Associated Virtual Reality Simulation System Based on MFC and Vega Software

Research and Simulation of Motion Model in Vega

Research and Design of Viewpoint Control Technology Based on Vega Platform

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

Study on Visual Function of Human Computer Interaction in Parachuting Simulator

An Electro-oculography Based Human-computer Interaction System

Research and Implementation of Real-time Visual Simulation System of Satellite Motion in Orbit

Design and Realization of Flight Simulation System Based on Virtual Reality Technology

Design of Virtual Scene Simulation System for Optoelectronic Tracking Equipment

Research on Designing and Realizing the Virtual Scene Using Vega Based on MFC

Real-time 3D Animated Flight Simulation Based on OpenGL

IR Flight Scene Simulation Based on Creator and Vega

Stroke-based Modeling and Haptic Skill Display for Chinese Calligraphy Simulation System

Interactive simulation of a 3d scene