Abstract:Taken it into consideration that human has a great deal of experiences and knowledge of hand postures, if these operating skills of postures are applied to HCI, the simple and convenient human-computer interface can be expected. In fact, tracking, recognition and interaction based on 3D freehand are a part of the cores in our virtual assembly system, but it is a challenging task to track 3D freehand in real-time because of high dimensionality of 3D full hand model. A novel framework for 3D freehand tracking is put forward in this paper. Firstly, we model and investigate this problem under our virtual assembly system (VAS), so as to decrease the arbitrariness and complexity of this issue. Secondly, we put emphasis on building cognitive and behavioral model (CBM) for users in VAS. Thirdly, we research on the way to track 3D freehand based on CBM. The main contributions of this paper are that we propose a new CBM, TPTM model, provide a way to connect users and computer for effective interaction, and present a real-time freehand tracking algorithm. Based on TPTM model, the prediction, the number of particles, the way and scope of sampling, are optimized. TPTM model not only explain behavioral characteristics for users but also can effectively guide the design of freehand tracking algorithm. TPTM model also provides a data structure that can facilitate the implementation of the tracking algorithm. Our experimental results show that the proposed approach raises the quality of each sampled particle or avoid sampling "poor" particles which appear with low probability in each frame, and it tracks 3D freehand in real-time with high accuracy. The number of the drawn particles is reduced up to 5 and the tracking speed increase up to 81 ms per frame.

Free-hand gesture control with “touchable” virtual interface for human-3DTV interaction

COMTIS: Customizable Touchless Interaction System for Large Screen Visualization

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Interaction and control with the auxiliary of hand gesture

3D Intuitive Gesture Interaction via Motion Sensing

RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch

A Motion-based User Interface for the Control of Virtual Humans Performing Sports.

An interaction system using mixed hand gestures.

An Interactive Control System for 3d Display Using Hidden Markov Model

Depth Camera-Based 3D Hand Gesture Controls with Immersive Tactile Feedback for Natural Mid-Air Gesture Interactions

Towards the Design of Effective Freehand Gestural Interaction for Interactive TV

A touchable virtual screen interaction system with handheld Kinect camera

Contactless Human–computer Interaction System Based on Three-Dimensional Holographic Display and Gesture Recognition

3D Articulated Hand Tracking Based on Behavioral Model

Design and evaluation of a hand gesture recognition approach for real-time interactions

Design and Development of a Gesture-Based Interaction System for Multi-Projector Tiled Display Wall

Motion-Based Perceptual User Interface

Hand-adaptive User Interface: Improved Gestural Interaction in Virtual Reality

Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors

A Practical Approach to Television-Assisted Control Utilizing Eye-Tracking Technology

Hand's 3D movement detection with one handheld camera.