Abstract:The uncertainty of hand gestures, the variability of gestures across subjects, and the high cost of collecting a large amount of annotated data lead to a great challenge to the robust recognition of gestures, and thus it remains quite crucial to capture the informative features of hand movements and to mitigate inter-subject variations. To this end, we propose a gesture recognition model that uses two different types of sensors and optimizes the feature space towards enhanced accuracy and better generalization. Specifically, we use an accelerometer and a surface electromyography sensor to capture kinematic and physiological signals of hand movements. We use a sliding window to divide the streaming sensor data and then extract time-domain and frequency-domain features from each segment to return feature vectors. Afterwards, the feature space is optimized with a feature selector and a gesture recognizer is optimized. To handle the case where no labeled training data are available for a new user, we apply the transfer learning technique to reuse the cross-subject knowledge. Finally, extensive comparative experiments concerning different classification models, different sensors, and different types of features are conducted. Results show that the joint use of kinematic and physiological sensors generally outperforms the use of single sensor, indicating the synthetic effect of different sensors, and that the use of transfer learning helps improve the cross-subject recognition accuracy. In addition, we quantitatively investigate the impact of null gesture on a gesture recognizer and results indicate that null gesture would lower its accuracy, enlightening related studies to consider it.

Multimodal Gesture Recognition for Mascot Robot System Based on Choquet Integral Using Camera and 3D Accelerometers Fusion

Multimodal Gesture Recognition Based On Choquet Integral

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM

Gesture Recognition with a 3-D Accelerometer

WristCam: A Wearable Sensor for Hand Trajectory Gesture Recognition and Intelligent Human–Robot Interaction

A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot

Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera

CAPG-MYO - A Muscle-Computer Interface Supporting User-defined Gesture Recognition.

Gesture Recognition Based on Localist Attractor Networks with Application to Robot Control [Application Notes]

Mascot Robot System Based on Fuzzy Control Technology

Natural Gesture Modeling and Recognition Approach Based on Joint Movements and Arm Orientations

A gesture recognition system using Localist Attractor Networks for human-robot interaction

Multi-modal learning for gesture recognition

Improving Hand Gesture Recognition Robustness to Dynamic Posture Variations by Multimodal Deep Feature Fusion

Multimodal Gesture Recognition Based on the ResC3D Network.

Research on Multimodal Human-Robot Interaction Based on Speech and Gesture.

Fusion of kinematic and physiological sensors for hand gesture recognition

A Multi-sensor Gesture Interaction System for Human-robot Cooperation

Multimodal information fusion for human-robot interaction

Research on Unified Recognition Model and Algorithm for Multi-Modal Gestures