Abstract:Compared with human-computer interaction in terms of optical and speech signals, inertial sensor-based handwriting recognition shows more competitive due to low power consumption, independence of environment, and portable wearing. However, recognizing handwriting characters under usual habits in continuous writing motion is quite challenging as it is hard to accurately extract data segments of characters from a continuous inertial data stream. Considering the cumbersome manual annotation of segmentation points in inertial data streams, we propose a trajectory upheaval network (TUN), which can be trained on unlabeled data and achieve character segmentation in successive writing motions. Furthermore, considering that individual writing habits lead to various stroke orders and ill-formed writing, we then propose a cooperation-competition learning (CCL) model, which fuses the features of writing trajectory morphology and the inertial dynamic signal, to recognize the semantics of these character motions with successes. The CCL guides a competing mechanism of the two features to avoid a collaborative degradation phenomenon while their mutual learning promotes the efficient fusion of two features to achieve optimal recognition performance. The extensive experiments illustrate that our TUN-CCL model achieves the final recognition accuracy of 98.6% for 62 English-number characters without requirements on stroke order and writing habits. The comparative experiment and ablation experiment validate the effectiveness of our method, which exhibits potential application prospects in human-computer interaction.

Machine-Learning Assisted Handwriting Recognition Using Graphene Oxide-Based Hydrogel

SmartHandwriting: Handwritten Chinese Character Recognition with Smartwatch.

SmartGe: Identifying Pen-Holding Gesture with Smartwatch.

SmartWriting: Pen-Holding Gesture Recognition with Smartwatch

Alphabet Handwriting Recognition: From Wood‐Framed Hydrogel Arrays Design to Machine Learning Decoding

Motion-Based Handwriting Recognition

Air-Writing Recognition Enabled by a Flexible Dual-Network Hydrogel-Based Sensor and Machine Learning

Self-Powered Intelligent Human-Machine Interaction for Handwriting Recognition.

MXene-Based Skin-Like Hydrogel Sensor and Machine Learning-Assisted Handwriting Recognition

Wearable One-Handed Keyboard Using Hydrogel-Based Mechanical Sensors for Human-Machine Interaction

Multifunctional, Self-Adhesive MXene-Based Hydrogel Flexible Strain Sensors for Hand-Written Digit Recognition with Assistance of Deep Learning

Machine Learning-Assisted Gesture Sensor Made with Graphene/Carbon Nanotubes for Sign Language Recognition

Gesture Recognition System Using Reduced Graphene Oxide-Enhanced Hydrogel Strain Sensors for Rehabilitation Training.

Handwriting Recognition under Natural Writing Habits based on A Low-cost Inertial Sensor

In Situ Polymerized MXene/Polypyrrole/Hydroxyethyl Cellulose-Based Flexible Strain Sensor Enabled by Machine Learning for Handwriting Recognition

Machine Learning-Enabled Intelligent Gesture Recognition and Communication System Using Printed Strain Sensors.

Multilanguage-handwriting self-powered recognition based on triboelectric nanogenerator enabled machine learning

A Wearable Real-Time Character Recognition System Based on Edge Computing-Enabled Deep Learning for Air-Writing

An adhesive, stretchable, and freeze-resistant conductive hydrogel strain sensor for handwriting recognition and depth motion monitoring

A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition

Deep Learning-Enabled Real-Time Personal Handwriting Electronic Skin with Dynamic Thermoregulating Ability