Abstract:Objetive. Decoding speech from brain activity can enable communication for individuals with speech disorders. Deep neural networks have shown great potential for speech decoding applications. However, the limited availability of large datasets containing neural recordings from speech-impaired subjects poses a challenge. Leveraging data from healthy participants can mitigate this limitation and expedite the development of speech neuroprostheses while minimizing the need for patient-specific training data. Approach. In this study, we collected a substantial dataset consisting of recordings from 56 healthy participants using 64 EEG channels. Multiple neural networks were trained to classify perceived sentences in the Spanish language using subject-independent, mixed-subjects, and fine-tuning approaches. The dataset has been made publicly available to foster further research in this area. Main results. Our results demonstrate a remarkable level of accuracy in distinguishing sentence identity across 30 classes, showcasing the feasibility of training Deep Neural Networks (DNNs) to decode sentence identity from perceived speech using EEG. Notably, the subject-independent approach rendered accuracy comparable to the mixed-subjects approach, although with higher variability among subjects. Additionally, our fine-tuning approach yielded even higher accuracy, indicating an improved capability to adapt to individual subject characteristics, which enhances performance. This suggests that DNNs have effectively learned to decode universal features of brain activity across individuals while also being adaptable to specific participant data. Furthermore, our analyses indicate that EEGNet and DeepConvNet exhibit comparable performance, outperforming ShallowConvNet for sentence identity decoding. Finally, our Grad-CAM visualization analysis identifies key areas influencing the network's predictions, offering valuable insights into the neural processes underlying language perception and comprehension. Significance. These findings advance our understanding of EEG-based speech perception decoding and hold promise for the development of speech neuroprostheses, particularly in scenarios where subjects cannot provide their own training data.

Efficient Automatic Speech Recognition from EEG Signals Using Optimal Deep Learning Approach

Silent Speech Recognition based on sEMG and EEG Signals

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

Aquila-Eagle-Based Deep Convolutional Neural Network for Speech Recognition Using EEG Signals

Imagined Speech Classification Using EEG and Deep Learning

Silent Speech Recognition using EEG Signals *

Brain Signals to Rescue Aphasia, Apraxia and Dysarthria Speech Recognition

Analysis of EEG frequency bands for Envisioned Speech Recognition

Optimal Deep Learning-Based Recognition Model for EEG Enabled Brain-Computer Interfaces Using Motor-Imagery

Recognition of words from brain-generated signals of speech-impaired people: Application of autoencoders as a neural Turing machine controller in deep neural networks

Predicting EEG Responses to Attended Speech via Deep Neural Networks for Speech

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

Speech2EEG: Leveraging Pretrained Speech Model for EEG Signal Recognition

Speech Emotion Recognition Using Deep Learning

Deep Learning-Based Approach for Emotion Recognition Using Electroencephalography (EEG) Signals Using Bi-Directional Long Short-Term Memory (Bi-LSTM)

Inner Speech Classification using EEG Signals: A Deep Learning Approach

Dementia Detection from Speech Using Machine Learning and Deep Learning Architectures

Design of EEG based thought identification system using EMD & deep neural network

Imagined speech classification exploiting EEG power spectrum features

EEG-based Subjects Identification based on Biometrics of Imagined Speech using EMD

Identification of perceived sentences using deep neural networks in EEG