Abstract:Hand gestures facilitate user interaction and immersion in virtual museum applications. These gestures allow users to navigate virtual exhibitions, interact with virtual artifacts, and control virtual environments naturally and intuitively. This study introduces a deep learning-driven approach for hand gesture recognition using wearable vision sensors designed for interactive virtual museum environments. The proposed approach employs an image-based feature extraction strategy that focuses on capturing five partial occupancy areas of the hand. Notably, a deep learning strategy using the bidirectional long short-term memory (Bi-LSTM) model is adopted to construct an effective model for hand gesture identification. The bidirectionality of Bi-LSTM enables it to capture dependencies in both forward and backward directions, providing a more comprehensive understanding of temporal relationships in the data. The bidirectional nature allows the model to better capture the dynamics and complexities of hand motions, leading to improved accuracy and robustness. The performance evaluation includes experiments on publicly available datasets, considering virtual and real museum scenarios. The results highlight the Bi-LSTM-based approach's superiority by accurately distinguishing various hand gestures. The experimental findings demonstrate that combining the five area ratios and Bi-LSTM classification enables robust recognition of diverse hand gestures and effectively discriminates between similar actions, such as slide left and right classes. Additionally, it shows promising detection performance compared to conventional machine learning models and state-of-the-art (SOTA) methods. The presented approach is promising for enhancing user interaction and immersion in virtual museum experiences.

Gesture recognition algorithm based on image information fusion in virtual reality

Improving Online Gesture Recognition with WarpingLCSS by Multi-Sensor Fusion

Dynamic hand gesture recognition using hidden Markov models

2D Motion Detection Bounded Hand 3D Trajectory Tracking and Gesture Recognition under Complex Background

Research and Application of Multifeature Gesture Recognition in Human-Computer Interaction Based on Virtual Reality Technology

Gesture Recognition with a 3-D Accelerometer

Gesture Recognition Algorithm Based on Computer Vision

Action recognition based on multimode fusion for VR online platform

Interaction and control with the auxiliary of hand gesture

Gesture recognition using combination of acceleration sensor and images for casual communication between robots and humans

Application of Multi-sensory interaction design Based on Machine Learning in Virtual Reality

Multi-sensors Based 3D Gesture Recognition and Interaction in Virtual Block Game

Real-Time Gesture Recognition with Virtual Glove Markers

An interaction system using mixed hand gestures.

Gesture-based target acquisition in virtual and augmented reality.

Deep Learning for Hand Gesture Recognition in Virtual Museum Using Wearable Vision Sensors

Application of Gesture Recognition Based on Spatiotemporal Graph Convolution Network in Virtual Reality Interaction

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Design and implementation of virtual fitting system based on gesture recognition and clothing transfer algorithm

Application of Artificial Intelligence in Hand Gesture Recognition with Virtual Reality: Survey and Analysis of Hand Gesture Hardware Selection

Design and exploration of a virtual reality-based human-computer interaction system