Abstract:Efficient typing or text-input on mobile devices, such as smartphones and wearables is a long-standing problem, due to the miniature touchscreen on the devices. Recently, touchscreen-free solutions leveraging on acoustic sensing have been proposed, with the advantage of low cost and ubiquitous availability. However, existing solutions usually require people to write in print-style, and more importantly, they are highly vulnerable to environmental change, i.e., they need repetitive training upon slight deviation of writing places or device locations. Therefore, they are far from practical usage. In this paper, we propose a novel acoustic-based text-input system called UbiWriter, which can recognize freestyle handwriting with high ubiquity, i.e., one-time training and writing elsewhere. UbiWriter is built on a new letter recognition principle, which treats the acoustic signal from writing a letter as a complete trajectory, and then distills the recognition feature that is resilient to environmental change. For the actual realization of the principle, we adopt and incorporate a series of techniques, including a feature-preserved fast letter alignment, ${K}$ -nearest neighbor letter classification, and language structure-driven word recognition. We also design and implement an APP with cloud-computing support, in order to facilitate real-time text input. Extensive experimental results demonstrate that UbiWriter outperforms the state-of-the-art under various practical settings.

WritePad: Consecutive Number Writing on Your Hand with Smart Acoustic Sensing

SmartWriting: Pen-Holding Gesture Recognition with Smartwatch

SmartHandwriting: Handwritten Chinese Character Recognition with Smartwatch.

SmartGe: Identifying Pen-Holding Gesture with Smartwatch.

AirText: One-Handed Text Entry in the Air for COTS Smartwatches

VPad: Virtual Writing Tablet for Laptops Leveraging Acoustic Signals

Write, Attend and Spell

HandPad: Make Your Hand an On-the-go Writing Pad Via Human Capacitance

Learning to Recognize Handwriting Input with Acoustic Features

SoundWrite: Text Input on Surfaces through Mobile Acoustic Sensing

AcouWrite: Acoustic-based Handwriting Recognition on Smartphones

Motion-Based Handwriting Recognition

A Rapid Adaptation Approach for Dynamic Air-Writing Recognition Using Wearable Wristbands with Self-Supervised Contrastive Learning

Handwriting Recognition System Leveraging Vibration Signal on Smartphones

Ipand: Accurate Gesture Input with Ambient Acoustic Sensing on Hand.

Ubiquitous Writer: Robust Text Input for Small Mobile Devices Via Acoustic Sensing.

WristSonic: Enabling Fine-grained Hand-Face Interactions on Smartwatches Using Active Acoustic Sensing

SonarWatch: Field sensing technique for smartwatches based on ultrasound and motion

New Application: A Hand Air Writing System Based on Radar Dual View Sequential Feature Fusion Idea.

Control with Gestures: A Hand Gesture Recognition System Using Off-the-Shelf Smartwatch.

WordRecorder: Accurate Acoustic-based Handwriting Recognition Using Deep Learning.