Abstract:This paper presents our recent development on a portable and refreshable text reading and sensory substitution system for the blind or visually impaired (BVI), called Finger-eye. The system mainly consists of an opto-text processing unit and a compact electro-tactile based display that can deliver text-related electrical signals to the fingertip skin through a wearable and Braille-dot patterned electrode array and thus delivers the electro-stimulation based Braille touch sensations to the fingertip. To achieve the goal of aiding BVI to read any text not written in Braille through this portable system, in this work, a Rapid Optical Character Recognition (R-OCR) method is firstly developed for real-time processing text information based on a Fisheye imaging device mounted at the finger-wearable electro-tactile display. This allows real-time translation of printed text to electro-Braille along with natural movement of user's fingertip as if reading any Braille display or book. More importantly, an electro-tactile neuro-stimulation feedback mechanism is proposed and incorporated with the R-OCR method, which facilitates a new opto-electrotactile feedback based text line tracking control approach that enables text line following by user fingertip during reading. Multiple experiments were designed and conducted to test the ability of blindfolded participants to read through and follow the text line based on the opto-electrotactile-feedback method. The experiments show that as the result of the opto-electrotactile-feedback, the users were able to maintain their fingertip within a $2mm$ distance of the text while scanning a text line. This research is a significant step to aid the BVI users with a portable means to translate and follow to read any printed text to Braille, whether in the digital realm or physically, on any surface.

[Recognition of Chinese Character on Electrotactile Vision Substitution System].

Braille-to-Chinese Translation System Based on Optical Braille Recognition

Design and Implementation of Tactile Vision Substitution System Based on the Fingertips

Development of an Eog (Electro-Oculography) Based Human-Computer Interface

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Recognition of pixelized Chinese characters using simulated prosthetic vision.

Seeing On The Surface Of Eyeballs: Non-Invasive Visual Substitution With The Electro-Tactile Device On Cornea

An Electro-oculography Based Human-computer Interaction System

Chinese Character Recognition Using Simulated Phosphene Maps

Vision-Braille: An End-to-End Tool for Chinese Braille Image-to-Text Translation

Mathematical morphology based electro-oculography recognition algorithm for human-computer interaction

Development of a character CAPTCHA recognition system for the visually impaired community using deep learning

A Virtual Chinese Keyboard BCI System Based on P300 Potentials

A Wearable Braille Recognition System Based on High Density Tactile Sensors

Eye tracking and eye expression decoding based on transparent, flexible and ultra-persistent electrostatic interface

A Virtual Keyboard Controlled By Eog

Printed Texts Tracking and Following for a Finger-Wearable Electro-Braille System Through Opto-electrotactile Feedback

Tactile Recognition of Chinese Character and Two-finger Pattern

[Recognition of Gaussian Pixelate Chinese Character Based on Simulated Prosthetic Vision].

Electronic Skin Based on PLLA/TFT/PVDF-TrFE Array for Multi-Functional Tactile Sensing and Visualized Restoring

Realization of Chinese characters input method based on embedded system