Abstract:Objective. Brain-computer interfaces aim to build an efficient communication with the world using neural signals, which may bring great benefits to human society, especially to people with physical impairments. To date, the ability to translate brain signals to effective communication outcome remains low. This work explores whether the handwriting process could serve as a potential interface with high performance. To this end, we first examined how much the scalp-recorded brain signals encode information related to handwriting and whether it is feasible to precisely retrieve the handwritten content solely from the scalp-recorded electrical data. Approach. Five participants were instructed to write the sentence 'HELLO, WORLD!' repeatedly on a tablet while their brain signals were simultaneously recorded by electroencephalography (EEG). The EEG signals were first decomposed by independent component analysis for extracting features to be used to train a convolutional neural network (CNN) to recognize the written symbols. Main results. The accuracy of the CNN-based classifier trained and applied on the same participant (training and test data separated) ranged from 76.8% to 97.0%. The accuracy of cross-participant application was more diverse, ranging from 14.7% to 58.7%. These results showed the possibility of recognizing the handwritten content directly from the scalp level brain signal. A demonstration of the recognition system in an online mode was presented. The major factor that grounded the recognition was the close association between the rich dynamics of electroencephalogram source activities and the kinematic information during the handwriting movements. Significance. This work revealed an explicit and precise mapping between scalp-level electrophysiological signals and linguistic information conveyed by handwriting, which provided a novel approach to developing brain computer interfaces that focus on semantic communication.

Tracking Children’s Handwriting Learning Process Using EEG: A System Development and Validation

SmartWriting: Pen-Holding Gesture Recognition with Smartwatch

Temporally resolved neural dynamics underlying handwriting

Effect of Handwriting on Visual Word Recognition in Chinese Bilingual Children and Adults

Neural Correlates of Handwriting Effects in L2 Learners

Reconstructing Multi-Stroke Characters from Brain Signals Toward Generalizable Handwriting Brain-Computer Interfaces

Brain Correlates of Chinese Handwriting and Their Relation to Reading Development in Children: An fMRI Study

Online recognition of handwritten characters from scalp-recorded brain activities during handwriting

Handwriting but not typewriting leads to widespread brain connectivity: a high-density EEG study with implications for the classroom

The Dynamics of Handwriting Improves the Automated Diagnosis of Dysgraphia

How beginning handwriting is influenced by letter knowledge: Visual-motor coordination during children's form copying

The influence of sensory-motor components of handwriting on Chinese character learning in second- and fourth-grade Chinese children.

Orthographic character complexity modulates dynamic neural activity in skilled handwriting

Evaluation of a group-based sensorimotor intervention programme to improve Chinese handwriting of primary school students

The brain basis of handwriting deficits in Chinese children with developmental dyslexia

Design and Development of an EOG-based Simplified Chinese Eye-Writing System

Towards Scalable Handwriting Communication via EEG Decoding and Latent Embedding Integration

Handwritten character classification from EEG through continuous kinematic decoding

Towards EEG-to-Text: Handwritten Character Classification via Continuous Kinematic Decoding

Enhancing Handwriting Performance of Children with Developmental Coordination Disorder (DCD) Using Computerized Visual Feedback

A statistical procedure to assist dysgraphia detection through dynamic modelling of handwriting