Abstract:Taken it into consideration that human has a great deal of experiences and knowledge of hand postures, if these operating skills of postures are applied to HCI, the simple and convenient human-computer interface can be expected. In fact, tracking, recognition and interaction based on 3D freehand are a part of the cores in our virtual assembly system, but it is a challenging task to track 3D freehand in real-time because of high dimensionality of 3D full hand model. A novel framework for 3D freehand tracking is put forward in this paper. Firstly, we model and investigate this problem under our virtual assembly system (VAS), so as to decrease the arbitrariness and complexity of this issue. Secondly, we put emphasis on building cognitive and behavioral model (CBM) for users in VAS. Thirdly, we research on the way to track 3D freehand based on CBM. The main contributions of this paper are that we propose a new CBM, TPTM model, provide a way to connect users and computer for effective interaction, and present a real-time freehand tracking algorithm. Based on TPTM model, the prediction, the number of particles, the way and scope of sampling, are optimized. TPTM model not only explain behavioral characteristics for users but also can effectively guide the design of freehand tracking algorithm. TPTM model also provides a data structure that can facilitate the implementation of the tracking algorithm. Our experimental results show that the proposed approach raises the quality of each sampled particle or avoid sampling "poor" particles which appear with low probability in each frame, and it tracks 3D freehand in real-time with high accuracy. The number of the drawn particles is reduced up to 5 and the tracking speed increase up to 81 ms per frame.

Markerless hand pose estimation system using Kinect

Markerless Motion Capture of Human Body Using PSO with Single Depth Camera

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

The Design and Implementation of Natural Human-Robot Interaction System Based on Kinect Sensor

Fast and Robust Hand Tracking Using Detection-Guided Optimization

Markerless Kinect-Based Hand Tracking for Robot Teleoperation

Performance Capture Of Interacting Characters With Handheld Kinect

A Survey on 3D Hand Pose Estimation: Cameras, Methods, and Datasets

Human Performance Capture Using Multiple Handheld Kinects

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

3D Articulated Hand Tracking Based on Behavioral Model

Combining Marker-based Mocap and RGB-D Camera for Acquiring High-fidelity Hand Motion Data.

Markerless Human–Manipulator Interface Using Leap Motion with Interval Kalman Filter and Improved Particle Filter

3-D Hand Pose Estimation from Kinect's Point Cloud Using Appearance Matching

A Markerless Human–Robot Interface Using Particle Filter and Kalman Filter for Dual Robots

A Markerless Human-Manipulators Interface Using Multi-Sensors.

Markerless Human-Manipulator Interface with Vibration Feedback Using Multi-sensors.

Markerless Motion Capture Using Appearance and Inertial Data

Locating human hands for real-time pose estimation from monocular video

Markerless human-robot interface for dual robot manipulators using Kinect sensor

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation