Abstract:In this paper, we present AccessWear, a system that improves the accessibility of smartphone touchscreen interactions for blind users using smartwatch gestures. Our system design is human-centered, namely, it incorporates the design goals that were learned from a formative user study with 9 blind participants. The formative study showed that blind users liked the idea of using smartwatch gestures as an alternative: 4 participants liked that when using smart-watch gestures, they did not have to bring their expensive phones out in public and 6 participants liked that smart-watch gestures can be performed with one-hand, as the other hand is usually occupied in holding a cane or a guide dog. Even though there are several advantages to smartwatch gestures, our study also shows that gestures performed by blind users have different patterns compared to sighted users, making gesture recognition more challenging. To this end, AccessWear makes two contributions. The first is a gesture recognition system that works specifically for blind users that is lightweight and does not require per-person training. The second is a near-zero-effort gesture replacement system that does not require any changes to the original application. AccessWear uses input virtualization techniques so that a given gesture can replace the touchscreen input seamlessly. We implement AccessWear on an Android smartphone and Android watch. We perform a quantitative and qualitative study with 8 blind participants. Our study shows that AccessWear can recognize gestures with a 92% accuracy and the end-to-end latency when using an alternate gesture was 53 msec on average. The qualitative study shows that when participants perform a task, consisting of a series of gestures, the system is robust, does not have perceived delays, and does not add physical or mental load on the users.

Keep the Phone in Your Pocket

EarTouch: Facilitating Smartphone Use for Visually Impaired People in Mobile and Public Scenarios

QwertyRing: Text Entry on Physical Surfaces Using a Ring

ARO: Exploring the Design of Smart-Ring Interactions for Encumbered Hands

iDial: Enabling a Virtual Dial Plate on the Hand Back for Around-Device Interaction.

Put a Ring on It: Text Entry Performance on a Grip Ring Attached Smartphone

AccessWear: Making Smartphone Applications Accessible to Blind Users

RingGesture: A Ring-Based Mid-Air Gesture Typing System Powered by a Deep-Learning Word Prediction Framework

Hold, Touch and Read It: Border Interactions in Mobile Reading Environment

Smart Rings vs. Smartwatches: Utilizing Motion Sensors for Gesture Recognition

3D Intuitive Gesture Interaction via Motion Sensing

picoRing: battery-free rings for subtle thumb-to-index input

PeriSense: Ring-Based Multi-Finger Gesture Interaction Utilizing Capacitive Proximity Sensing

DRG-Keyboard: Enabling Subtle Gesture Typing on the Fingertip with Dual IMU Rings.

Ring-a-Pose: A Ring for Continuous Hand Pose Tracking

FootUI: Assisting People with Upper Body Motor Impairments to Use Smartphones with Foot Gestures on the Bed

Designing and Evaluating Hand-to-Hand Gestures with Dual Commodity Wrist-Worn Devices

Wearable Smart Rings for Multifinger Gesture Recognition Using Supervised Learning