BrailleReader : Braille Character Recognition Using Wearable Motion Sensor

Xiaochen Liu,Fan Li,Yetong Cao,Yu Wang

DOI: https://doi.org/10.1109/tmc.2024.3379569

IF: 6.075

2024-01-01

IEEE Transactions on Mobile Computing

Abstract:With the ever-increasing demand for improving communication and independence for visually impaired people, automatic Braille recognition has gained increasing attention in facilitating Braille learning and reading. However, current approaches mainly require high-cost hardware, involve inconvenient operation, and disturb the normal touch function. In this paper, we propose BrailleReader as a low-cost and effortless Braille character recognition system without disturbing normal Braille touching. It exploits the wrist motion of Braille reading captured by the motion sensor available in the ubiquitous wrist-worn device to infer the encoded character information. To address the noise caused by other body and hand movements, we propose a novel noise cancellation method using the wavelet packet decomposition and reconstruction technique to separate clean wrist movement induced by the Braille dot. Moreover, we further explore the unique wrist movement pattern in three aspects to extract a novel and effective feature set. Based on this, BrailleReader leverages a spiking neural network-based model to robustly recognize Braille characters across different people and different surface materials. Extensive experiments with 48 participants demonstrate that BrailleReader can perform accurate and robust recognition of 26 Braille characters.

computer science, information systems,telecommunications