Abstract:Gestures serve an important role in enabling natural interactions with computing devices, and they form an important part of everyday nonverbal communication. In increasingly many application scenarios of gesture interaction, such as gesture-based authentication, calligraphy, sketching, and even artistic expression, not only are the underlying gestures complex and consist of multiple strokes but also the correctness of the gestures depends on the order at which the strokes are performed. In this paper, we present WiCG, an innovative and novel WiFi sensing approach for capturing and providing feedback on stroke order. Our approach tracks the user’s hand movement during writing and exploits this information in combination with statistical methods and machine learning techniques to infer what characters have been written and at which stroke order. We consider Chinese calligraphy as our use case as the resulting gestures are highly complex, and their assessment depends on the correct stroke order. We develop a set of analyses and algorithms to overcome many issues of this challenging task. We have conducted extensive experiments and user studies to evaluate our approach. Experimental results show that our approach is highly effective in identifying the written characters and their written stroke order. We show that our approach can adapt to different deployment environments and user patterns.

Separating and Recognizing Gestural Strokes for Sketch-Based Interfaces

Pen-based gesture design and recognition for sketch editing

An online sketching and gesture editing system for conceptual design

SmartWriting: Pen-Holding Gesture Recognition with Smartwatch

Research and Implementation of the Sketching Design System

SmartGe: Identifying Pen-Holding Gesture with Smartwatch.

Sketching Feature-Based Modeling by Capturing Design Intention

A Multitouch Drawing Application With Occlusion-Free Interaction Strategies

Model-Guided 3D Sketching

On-Line Sketch Recognition Using Direction Feature

Segmentation of Online Sketching Using Geometric Feature

Interactively Painting by Shu-Kong Facsimile Based on Motion Sensors

Online Sketch Recognition Using Geometric-Based Classifiers

User-Defined Gestures for Gestural Interaction: Extending from Hands to Other Body Parts

Controlling Non-Touch Screens as Touch Screens Using Airpen, a Writing Tool with In-Air Gesturing Mode

3D Intuitive Gesture Interaction via Motion Sensing

Exploiting Serialized Fine-Grained Action Recognition Using WiFi Sensing

GestureSurface: VR Sketching Through Assembling Scaffold Surface with Non-Dominant Hand.

A Freehand Sketchy Graphic Input System: SketchGIS

Sketching Interfaces for Remote Collaboration

Freehand Sketching Interfaces: Early Processing For Sketch Recognition