Abstract:Air-writing is a challenging task that combines the fields of computer vision and natural language processing, offering an intuitive and natural approach for human-computer interaction. However, current air-writing solutions face two primary challenges: (1) their dependency on complex sensors (e.g., Radar, EEGs and others) for capturing precise handwritten trajectories, and (2) the absence of a video-based air-writing dataset that covers a comprehensive vocabulary range. These limitations impede their practicality in various real-world scenarios, including the use on devices like iPhones and laptops. To tackle these challenges, we present the groundbreaking air-writing Chinese character video dataset (AWCV-100K-UCAS2024), serving as a pioneering benchmark for video-based air-writing. This dataset captures handwritten trajectories in various real-world scenarios using commonly accessible RGB cameras, eliminating the need for complex sensors. AWCV-100K-UCAS2024 includes 8.8 million video frames, encompassing the complete set of 3,755 characters from the GB2312-80 level-1 set (GB1). Furthermore, we introduce our baseline approach, the video-based character recognizer (VCRec). VCRec adeptly extracts fingertip features from sparse visual cues and employs a spatio-temporal sequence module for analysis. Experimental results showcase the superior performance of VCRec compared to existing models in recognizing air-written characters, both quantitatively and qualitatively. This breakthrough paves the way for enhanced human-computer interaction in real-world contexts. Moreover, our approach leverages affordable RGB cameras, enabling its applicability in a diverse range of scenarios. The code and data examples will be made public at <a class="link-external link-https" href="https://github.com/wmeiqi/AWCV" rel="external noopener nofollow">this https URL</a>.

Construction of an interactive input and recognition experimental platform for handw ritten Chinese characters in air

Finger-writing-in-the-air system using Kinect sensor.

A New Writing Experience: Finger Writing in the Air Using a Kinect Sensor.

Reconstructing Multi-Stroke Characters from Brain Signals Toward Generalizable Handwriting Brain-Computer Interfaces

Real-time fingertip tracking and detection using Kinect depth sensor for a new writing-in-the air system

Hidden Markov random field based approach for off-line handwritten Chinese character recognition

Design and implementation of an algorithm for online recognition of hand-written Chinese characters

A New Hybrid-Parameter Recurrent Neural Network for Online Handwritten Chinese Character Recognition

A Video-Based Online Handwritten Chinese Character Recognition Method

Design of Handwritten Chinese Character Input System Based on Real-time Sample Collection

Motion-Based Handwriting Recognition

Online and offline handwritten Chinese character recognition

Hybrid Recognition for One Stroke Style Cursive Handwriting Characters

An Interactive Oral Training Platform Based on Kinect for EFL Learning

Chinese Character Learning System

Finger in Camera Speaks Everything: Unconstrained Air-Writing for Real-World

Online Handwriting Recognition Based on Microphone and IMU

AirContour

AirWrite: an Aerial Handwriting Trajectory Tracking and Recognition System with Mmwave

Use of Automatic Chinese Character Decomposition and Human Gestures for Chinese Calligraphy Robots

Online Gesture-Based Interaction with Visual Oriental Characters Based on Manifold Learning.