G3-2d: A Continuous Gesture Segmentation and Recognition Model to Shoot Gestures from 3d Space to 2d Coordinate System

Zijian Li,Yajun Zhang,Zhixiong Yang,Xu Liu,Bo Yuan
DOI: https://doi.org/10.2139/ssrn.4234143
2022-01-01
Abstract:As a new research hotspot in computer vision, gesture recognition has wide applications in smart home, virtual reality, robot control and so on. However, current CV-based gesture recognition techniques are mainly used to recognise static gestures, so in this paper, we propose G3-2D, a model that uses a novel approach to recognise continuous dynamic gestures by using a YoloV5 neural network that introduces an attention mechanism to detect gesture position coordinates and plot the gesture trajectory in a picture based on the captured position coordinates, thus mapping the 3D complex space of In order to remove the interference of other people's gestures, we propose an outlier removal algorithm, and to achieve segmentation of continuous gestures, we also propose a time window-based model to obtain the beginning and end of a single gesture. . Finally, the segmented gesture trajectory images are fed into a convolutional neural network for recognition. The system eventually achieves recognition of numbers and 26 English letters, and experimental results demonstrate the effectiveness of the model with an average accuracy of 94.3% and 92.2% for single dynamic gestures and continuous dynamic gestures, respectively.
What problem does this paper attempt to address?