Handwriting Recognition under Natural Writing Habits based on A Low-cost Inertial Sensor

Yifeng Wang,Yi Zhao

DOI: https://doi.org/10.1109/jsen.2023.3331011

IF: 4.3

2023-01-01

IEEE Sensors Journal

Abstract:Compared with human-computer interaction in terms of optical and speech signals, inertial sensor-based handwriting recognition shows more competitive due to low power consumption, independence of environment, and portable wearing. However, recognizing handwriting characters under usual habits in continuous writing motion is quite challenging as it is hard to accurately extract data segments of characters from a continuous inertial data stream. Considering the cumbersome manual annotation of segmentation points in inertial data streams, we propose a trajectory upheaval network (TUN), which can be trained on unlabeled data and achieve character segmentation in successive writing motions. Furthermore, considering that individual writing habits lead to various stroke orders and ill-formed writing, we then propose a cooperation-competition learning (CCL) model, which fuses the features of writing trajectory morphology and the inertial dynamic signal, to recognize the semantics of these character motions with successes. The CCL guides a competing mechanism of the two features to avoid a collaborative degradation phenomenon while their mutual learning promotes the efficient fusion of two features to achieve optimal recognition performance. The extensive experiments illustrate that our TUN-CCL model achieves the final recognition accuracy of 98.6% for 62 English-number characters without requirements on stroke order and writing habits. The comparative experiment and ablation experiment validate the effectiveness of our method, which exhibits potential application prospects in human-computer interaction.

engineering, electrical & electronic,instruments & instrumentation,physics, applied