Abstract:In this paper, we present AccessWear, a system that improves the accessibility of smartphone touchscreen interactions for blind users using smartwatch gestures. Our system design is human-centered, namely, it incorporates the design goals that were learned from a formative user study with 9 blind participants. The formative study showed that blind users liked the idea of using smartwatch gestures as an alternative: 4 participants liked that when using smart-watch gestures, they did not have to bring their expensive phones out in public and 6 participants liked that smart-watch gestures can be performed with one-hand, as the other hand is usually occupied in holding a cane or a guide dog. Even though there are several advantages to smartwatch gestures, our study also shows that gestures performed by blind users have different patterns compared to sighted users, making gesture recognition more challenging. To this end, AccessWear makes two contributions. The first is a gesture recognition system that works specifically for blind users that is lightweight and does not require per-person training. The second is a near-zero-effort gesture replacement system that does not require any changes to the original application. AccessWear uses input virtualization techniques so that a given gesture can replace the touchscreen input seamlessly. We implement AccessWear on an Android smartphone and Android watch. We perform a quantitative and qualitative study with 8 blind participants. Our study shows that AccessWear can recognize gestures with a 92% accuracy and the end-to-end latency when using an alternate gesture was 53 msec on average. The qualitative study shows that when participants perform a task, consisting of a series of gestures, the system is robust, does not have perceived delays, and does not add physical or mental load on the users.

Demo: UIWear

Uiwear: Easily Adapting User Interfaces For Wearable Devices

Wearable-based Human-Computer Interaction with LimbMotion

Demo Abstract: Wearablehub: an Open Pervasive Wearable Data Fusion Platform for Personal Health Management

A Survey of Patterns for Adapting Smartphone App UIs to Smart Watches.

Wearable Internet of Things: Concept, Architectural Components and Promises for Person-Centered Healthcare

Interaction Proxy Manager: Semantic Model Generation and Run-time Support for Reconstructing Ubiquitous User Interfaces of Mobile Services.

MagicWatch: interacting & segueing.

SimUser: Generating Usability Feedback by Simulating Various Users Interacting with Mobile Applications

An Adaptive Agent-Based Interface for Personalized Health Interventions

Robust Finger Interactions with COTS Smartwatches via Unsupervised Siamese Adaptation

Motion-Based Perceptual User Interface

AccessWear: Making Smartphone Applications Accessible to Blind Users

Push-button Synthesis of Watch Companions for Android Apps

Interaction Proxy Manager

Software for Wearable Devices: Challenges and Opportunities

A stand-alone proximity-based gaming wearable for remote physical activity monitoring

Sensor Virtualization for Efficient Sharing of Mobile and Wearable Sensors.

Chat-to-Design: AI Assisted Personalized Fashion Design

Lightful user interaction on smart wearables

Demo of Headbanger: Authenticating Smart Wearable Devices Using Unique Head Movement Patterns