Abstract:Background: Hand gesture interfaces are dedicated programs that principally perform hand tracking and hand gesture prediction to provide alternative controls and interaction methods. They take advantage of one of the most natural ways of interaction and communication, proposing novel input and showing great potential in the field of the human-computer interaction. Developing a flexible and rich hand gesture interface is known to be a time-consuming and arduous task. Previously published studies have demonstrated the significance of the finite-state-machine (FSM) approach when mapping detected gestures to GUI actions. Methods: In our hand gesture interface, we broadened the FSM approach by utilizing gesture-specific attributes, such as distance between hands, distance from the camera, and time of occurrences, to enable users to perform unique GUI actions. These attributes are obtained from hand gestures detected by the RealSense SDK employed in our hand gesture interface. By means of these gesture-specific attributes, users can activate static gestures and perform them as dynamic gestures. We also provided supplementary features to enhance the efficiency, convenience, and user-friendliness of our hand gesture interface. Moreover, we developed a complementary application for recording hand gestures by capturing hand keypoints in depth and color images to facilitate the generation of hand gesture datasets. Results: We conducted a small-scale user survey with fifteen subjects to test and evaluate our hand gesture interface. Anonymous feedback obtained from the users indicates that our hand gesture interface is adequately facile and self-explanatory to use. In addition, we received constructive feedback about minor flaws regarding the responsiveness of the interface. Conclusions: We proposed a hand gesture interface along with key concepts to attain user-friendliness and effectiveness in the control of existing GUIs.

Review on Perceptual User Interfaces Based on Motion Sensors

Motion-Based Perceptual User Interface

A Perceptual User Interface For Motion Retrieval

3D Intuitive Gesture Interaction via Motion Sensing

Review of Sensory Feedback Simulation Methods in Multi-modal Human-computer Interaction

Perceiving Motion Cues Inspired by Microsoft Kinect Sensor on Game Experiencing

PERCEPTUAL USER INTERFACE IN VIRTUAL SHOPPING ENVIRONMENT

Review of Non-Intrusive Interaction Based on Camera

Recent Progress of Tactile and Force Sensors for Human-Machine Interaction

The Influence of User-Tracking Feedback Format on Gestural Interaction's User Experience:A Kinect-Based Usability Study

Intuitive User Interface for Mobile Devices Based on Visual Motion Detection

Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller

Peye: Toward A Visual Motion Based Perceptual Interface For Mobile Devices

Promoted Motion-Sensing Gestural Interaction by Adapting to Group Users' Spatial Movements

A Review of Multimodal Interaction

A Quick Review of Human Perception in Immersive Media

User interface based on natural interaction design for seniors

Technologies toward next generation human machine interfaces: From machine learning enhanced tactile sensing to neuromorphic sensory systems

When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors from the Signal Processing Perspective

Eye Tracking Based Control System for Natural Human-Computer Interaction

On Attaining User-Friendly Hand Gesture Interfaces to Control Existing GUIs.