Abstract:Neurological disorders affecting speech production adversely impact quality of life for over 7 million individuals in the US. Traditional speech interfaces like eyetracking devices and P300 spellers are slow and unnatural for these patients. An alternative solution, speech Brain-Computer Interfaces (BCIs), directly decodes speech characteristics, offering a more natural communication mechanism. This research explores the feasibility of decoding speech features using non-invasive EEG. Nine neurologically intact participants were equipped with a 63-channel EEG system with additional sensors to eliminate eye artifacts. Participants read aloud sentences displayed on a screen selected for phonetic similarity to the English language. Deep learning models, including Convolutional Neural Networks and Recurrent Neural Networks with and without attention modules, were optimized with a focus on minimizing trainable parameters and utilizing small input window sizes for real-time application. These models were employed for discrete and continuous speech decoding tasks, achieving statistically significant participant-independent decoding performance for discrete classes and continuous characteristics of the produced audio signal. A frequency sub-band analysis highlighted the significance of certain frequency bands (delta, theta, and gamma) for decoding performance, and a perturbation analysis was used to identify crucial channels. Assessed channel selection methods did not significantly improve performance, suggesting a distributed representation of speech information encoded in the EEG signals. Leave-One-Out training demonstrated the feasibility of utilizing common speech neural correlates, reducing data collection requirements from individual participants.

Continuous Silent Speech Recognition using EEG

Speech Recognition With No Speech Or With Noisy Speech Beyond English

[The Neural Encoding of Continuous Speech - Recent Advances in EEG and MEG Studies].

Improving EEG based Continuous Speech Recognition

EEG based Continuous Speech Recognition using Transformers

Silent Speech Recognition based on sEMG and EEG Signals

Understanding effect of speech perception in EEG based speech recognition systems

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

Speech Recognition using EEG signals recorded using dry electrodes

Continuous and discrete decoding of overt speech with electroencephalography

Silent Speech Recognition using EEG Signals *

Relating EEG to continuous speech using deep neural networks: a review

Silent EEG Classification Using Cross-Fusion Adaptive Graph Convolution Network for Multilingual Neurolinguistic Signal Decoding.

Improving EEG based continuous speech recognition using GAN

Inner speech recognition through electroencephalographic signals

Brain Signals to Rescue Aphasia, Apraxia and Dysarthria Speech Recognition

GCN-LSTM for EEG Classification Based on Unspoken Speech of Bilinguals.

Speech Synthesis using EEG

Inner Speech Classification using EEG Signals: A Deep Learning Approach

Decoding Silent Reading EEG Signals Using Adaptive Feature Graph Convolutional Network