Abstract:The uncertainty of hand gestures, the variability of gestures across subjects, and the high cost of collecting a large amount of annotated data lead to a great challenge to the robust recognition of gestures, and thus it remains quite crucial to capture the informative features of hand movements and to mitigate inter-subject variations. To this end, we propose a gesture recognition model that uses two different types of sensors and optimizes the feature space towards enhanced accuracy and better generalization. Specifically, we use an accelerometer and a surface electromyography sensor to capture kinematic and physiological signals of hand movements. We use a sliding window to divide the streaming sensor data and then extract time-domain and frequency-domain features from each segment to return feature vectors. Afterwards, the feature space is optimized with a feature selector and a gesture recognizer is optimized. To handle the case where no labeled training data are available for a new user, we apply the transfer learning technique to reuse the cross-subject knowledge. Finally, extensive comparative experiments concerning different classification models, different sensors, and different types of features are conducted. Results show that the joint use of kinematic and physiological sensors generally outperforms the use of single sensor, indicating the synthetic effect of different sensors, and that the use of transfer learning helps improve the cross-subject recognition accuracy. In addition, we quantitatively investigate the impact of null gesture on a gesture recognizer and results indicate that null gesture would lower its accuracy, enlightening related studies to consider it.

Data fusion-based real-time hand gesture recognition with Kinect V2

Dynamic hand gesture recognition using hidden Markov models

Improving Online Gesture Recognition with WarpingLCSS by Multi-Sensor Fusion

A real-time hand gesture recognition method

Research on Dynamic Gesture Recognition with Low-Pixel ToF-Sensors

Real-Time Hand Gesture Recognition Using RGB-D Sensor

Kinect Sensor-Based Long-Distance Hand Gesture Recognition and Fingertip Detection with Depth Information

Fusion of kinematic and physiological sensors for hand gesture recognition

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Real-Time Hand Gesture Recognition: Integrating Skeleton-Based Data Fusion and Multi-Stream CNN

Multimodal Gesture Recognition Based on Attention Slow-Fast Fusion Networks

Fast and Robust Dynamic Hand Gesture Recognition via Key Frames Extraction and Feature Fusion

Dynamic gesture recognition based on 2D convolutional neural network and feature fusion

Stable And Real-Time Hand Gesture Recognition Based On Rgb-D Data

Online human gesture recognition from motion data streams.

A Real Time Vision-Based Hand Gestures Recognition System

Real-Time Gesture Recognition with Virtual Glove Markers

Multi-modal fusion for robust hand gesture recognition based on heterogeneous networks

Quantum nondemolition measurement of the photon number via the optical Kerr effect.

A Multi-sensor Gesture Interaction System for Human-robot Cooperation

Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction