Semantic hand gesture integration system using self-co-articulation and movement epenthesis detection

Shweta Saboo,Joyeeta Singha
DOI: https://doi.org/10.1007/s00371-024-03394-3
IF: 2.835
2024-05-08
The Visual Computer
Abstract:Recognizing hand gestures poses a formidable challenge, particularly when dealing with semantic gestures that require disentanglement prior to recognition. This paper addresses the intricate issue of an additional stroke, commonly referred to as 'movement epenthesis stroke,' which emerges between continuous gestures. Our proposed system employs a multifaceted approach to tackle this challenge. Initially, the system extracts color-motion information to facilitate hand detection, subsequently employing a fusion of shape information and a modified Kanade–Lucas–Tomasi (KLT) feature tracker. This integration significantly mitigates the issue of occlusions. The identification of movement epenthesis is accomplished by analyzing the gesture trajectory using a speed profile. Furthermore, self-co-articulation strokes are discerned by leveraging slope-angle information. To enhance the recognition process, a carefully selected set of 40 features is extracted, which are then employed for recognizing the resulting meaningful gestures. These features serve as inputs to various classification models, including support vector machines (SVM), k-nearest neighbors (kNN), and extreme learning machines (ELM). Deep learning algorithms are judiciously deployed to recognize gesture trajectories, thus streamlining the time-consuming feature extraction process. The outcomes of individual classifiers are amalgamated, resulting in a classifier fusion model. This model is enhanced through majority voting and is used in conjunction with cross-validation results. The experimental analysis culminates in an impressive accuracy rate of 98.88% achieved by the classifier fusion model. This achievement surpasses the performance of individual classifiers, underscoring the effectiveness of our proposed methodology.
computer science, software engineering
What problem does this paper attempt to address?