Abstract:We propose HeadGesture, a hands-free input approach to interact with Head Mounted Display (HMD) devices. Using HeadGesture, users do not need to raise their arms to perform gestures or operate remote controllers in the air. Instead, they perform simple gestures with head movement to interact with the devices. In this way, users' hands are free to perform other tasks, e.g., taking notes or manipulating tools. This approach also reduces the hand occlusion of the field of view [11] and alleviates arm fatigue [7]. However, one main challenge for HeadGesture is to distinguish the defined gestures from unintentional movements. To generate intuitive gestures and address the issue of gesture recognition, we proceed through a process of Exploration - Design - Implementation - Evaluation. We first design the gesture set through experiments on gesture space exploration and gesture elicitation with users. Then, we implement algorithms to recognize the gestures, including gesture segmentation, data reformation and unification, feature extraction, and machine learning based classification. Finally, we evaluate user performance of HeadGesture in the target selection experiment and application tests. The results demonstrate that the performance of HeadGesture is comparable to mid-air hand gestures, measured by completion time. Additionally, users feel significantly less fatigue than when using hand gestures and can learn and remember the gestures easily. Based on these findings, we expect HeadGesture to be an efficient supplementary input approach for HMD devices.

FaceSight: Enabling Hand-to-Face Gesture Interaction on AR Glasses with a Downward-Facing Camera Vision

TeethFa: Real-time, Hand-Free Teeth Gestures Interaction Using Fabric Sensors

A Bare Hand Controlled Ar Map Navigation System

Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera

WristCam: A Wearable Sensor for Hand Trajectory Gesture Recognition and Intelligent Human–Robot Interaction

Enabling Voice-Accompanying Hand-to-Face Gesture Recognition with Cross-Device Sensing

Hands-Free User Interface for AR/VR Devices Exploiting Wearer's Facial Gestures Using Unsupervised Deep Learning

iFace: Hand-Over-Face Gesture Recognition Leveraging Impedance Sensing

HandSee

Gaze Gestures and Their Applications in human-computer interaction with a head-mounted display

Hand Gestures Recognition in Videos Taken with Lensless Camera

Hand gestures recognition in videos taken with a lensless camera

Exploring Gaze Movement Gesture Recognition Method for Eye-Based Interaction Using Eyewear with Infrared Distance Sensor Array

HeadGesture

AUGlasses: Continuous Action Unit based Facial Reconstruction with Low-power IMUs on Smart Glasses

Hands-Free User Interface for VR Headsets Based on In Situ Facial Gesture Sensing

Facial performance sensing head-mounted display

Airfinger: Micro Finger Gesture Recognition Via NIR Light Sensing for Smart Devices

Hand Gesture Recognition using Deep Feature Fusion Network based on Wearable Sensors

Sensor-Assisted Multi-View Face Recognition System on Smart Glass

Helios: An extremely low power event-based gesture recognition for always-on smart eyewear