Abstract:Designing an extensive set of mid-air gestures that are both easy to learn and perform quickly presents a significant challenge. Further complicating this challenge is achieving high-accuracy detection of such gestures using commonly available hardware, like a 2D commodity camera. Previous work often proposed smaller, application-specific gesture sets, requiring specialized hardware and struggling with adaptability across diverse environments. Addressing these limitations, this paper introduces Abacus Gestures, a comprehensive collection of 100 mid-air gestures. Drawing on the metaphor of Finger Abacus counting, gestures are formed from various combinations of open and closed fingers, each assigned different values. We developed an algorithm using an off-the-shelf computer vision library capable of detecting these gestures from a 2D commodity camera feed with an accuracy exceeding 98% for palms facing the camera and 95% for palms facing the body. We assessed the detection accuracy, ease of learning, and usability of these gestures in a user study involving 20 participants. The study found that participants could learn Abacus Gestures within five minutes after executing just 15 gestures and could recall them after a four-month interval. Additionally, most participants developed motor memory for these gestures after performing 100 gestures. Most of the gestures were easy to execute with the designated finger combinations, and the flexibility in executing the gestures using multiple finger combinations further enhanced the usability. Based on these findings, we created a taxonomy that categorizes Abacus Gestures into five groups based on motor memory development and three difficulty levels according to their ease of execution. Finally, we provided design guidelines and proposed potential use cases for Abacus Gestures in the realm of mid-air interaction.

An empirical evaluation for defining a mid-air gesture dictionary for web-based interaction

Exploring Mid-Air Hand Interaction in Data Visualization

Differences in muscle activity, kinematics, user performance, and subjective assessment between touchscreen and mid-air interactions on a tablet

User-Defined Gestures for Gestural Interaction: Extending from Hands to Other Body Parts

Which One to Go: Security and Usability Evaluation of Mid-Air Gestures.

Working Memory Capacity for Mid-Air Gestures in Human–Computer Interaction

Mid-air gestures for in-vehicle media player: elicitation, segmentation, recognition, and eye-tracking testing

Elicitation and Evaluation of Hand-based Interaction Language for 3D Conceptual Design in Mixed Reality

Abacus Gestures

3D Intuitive Gesture Interaction via Motion Sensing

Human Factor Studies on Gestural Interaction: Past, Present, and Future

Web-based Interactive Drone Control Using Hand Gesture

Depth and Breadth of Pie Menus for Mid-air Gesture Interaction

Understanding the Design Space of Mouth Microgestures

Defining and Analyzing a Gesture Set for Interactive TV Remote on Touchscreen Phones

Evaluation of Full-Body Gestures Performed by Individuals with Down Syndrome: Proposal for Designing User Interfaces for All Based on Kinect Sensor

Study on the Design of a Non-Contact Interaction System Using Gestures: Framework and Case Study

Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation Studies

Gesture-based Human-Machine Interaction: Taxonomy, Problem Definition, and Analysis

AirWare: Utilizing Embedded Audio and Infrared Signals for In-Air Hand-Gesture Recognition

Evaluating Data-Driven Co-Speech Gestures of Embodied Conversational Agents through Real-Time Interaction