Abstract:Objetive. Decoding speech from brain activity can enable communication for individuals with speech disorders. Deep neural networks have shown great potential for speech decoding applications. However, the limited availability of large datasets containing neural recordings from speech-impaired subjects poses a challenge. Leveraging data from healthy participants can mitigate this limitation and expedite the development of speech neuroprostheses while minimizing the need for patient-specific training data. Approach. In this study, we collected a substantial dataset consisting of recordings from 56 healthy participants using 64 EEG channels. Multiple neural networks were trained to classify perceived sentences in the Spanish language using subject-independent, mixed-subjects, and fine-tuning approaches. The dataset has been made publicly available to foster further research in this area. Main results. Our results demonstrate a remarkable level of accuracy in distinguishing sentence identity across 30 classes, showcasing the feasibility of training Deep Neural Networks (DNNs) to decode sentence identity from perceived speech using EEG. Notably, the subject-independent approach rendered accuracy comparable to the mixed-subjects approach, although with higher variability among subjects. Additionally, our fine-tuning approach yielded even higher accuracy, indicating an improved capability to adapt to individual subject characteristics, which enhances performance. This suggests that DNNs have effectively learned to decode universal features of brain activity across individuals while also being adaptable to specific participant data. Furthermore, our analyses indicate that EEGNet and DeepConvNet exhibit comparable performance, outperforming ShallowConvNet for sentence identity decoding. Finally, our Grad-CAM visualization analysis identifies key areas influencing the network's predictions, offering valuable insights into the neural processes underlying language perception and comprehension. Significance. These findings advance our understanding of EEG-based speech perception decoding and hold promise for the development of speech neuroprostheses, particularly in scenarios where subjects cannot provide their own training data.

Decoding Imagined Speech from EEG Data: A Hybrid Deep Learning Approach to Capturing Spatial and Temporal Features

Decoding imagined speech from EEG signals using hybrid-scale spatial-temporal dilated convolution network

Imagined Speech Classification Using EEG and Deep Learning

A Novel Deep Learning Architecture for Decoding Imagined Speech from EEG

Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals

Hierarchical Deep Feature Learning For Decoding Imagined Speech From EEG

Decoding High-level Imagined Speech using Attention-based Deep Neural Networks

Electroencephalogram (EEG) Based Imagined Speech Decoding and Recognition

Decoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals

Speech decoding from stereo-electroencephalography (sEEG) signals using advanced deep learning methods

Imagined speech can be decoded from low- and cross-frequency intracranial EEG features

Imagined speech classification exploiting EEG power spectrum features

Towards Imagined Speech: Identification of Brain States from EEG Signals for BCI-based Communication Systems

Inner Speech Classification using EEG Signals: A Deep Learning Approach

Delineating neural contributions to electroencephalogram-based speech decoding

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

Identification of perceived sentences using deep neural networks in EEG

A Novel Hybrid Decoding Neural Network for EEG Signal Representation

Recognition of EEG Signals from Imagined Vowels Using Deep Learning Methods

Continuous and discrete decoding of overt speech with electroencephalography

Decoding Imagined Speech and Computer Control using Brain Waves