Abstract:Objetive. Decoding speech from brain activity can enable communication for individuals with speech disorders. Deep neural networks have shown great potential for speech decoding applications. However, the limited availability of large datasets containing neural recordings from speech-impaired subjects poses a challenge. Leveraging data from healthy participants can mitigate this limitation and expedite the development of speech neuroprostheses while minimizing the need for patient-specific training data. Approach. In this study, we collected a substantial dataset consisting of recordings from 56 healthy participants using 64 EEG channels. Multiple neural networks were trained to classify perceived sentences in the Spanish language using subject-independent, mixed-subjects, and fine-tuning approaches. The dataset has been made publicly available to foster further research in this area. Main results. Our results demonstrate a remarkable level of accuracy in distinguishing sentence identity across 30 classes, showcasing the feasibility of training Deep Neural Networks (DNNs) to decode sentence identity from perceived speech using EEG. Notably, the subject-independent approach rendered accuracy comparable to the mixed-subjects approach, although with higher variability among subjects. Additionally, our fine-tuning approach yielded even higher accuracy, indicating an improved capability to adapt to individual subject characteristics, which enhances performance. This suggests that DNNs have effectively learned to decode universal features of brain activity across individuals while also being adaptable to specific participant data. Furthermore, our analyses indicate that EEGNet and DeepConvNet exhibit comparable performance, outperforming ShallowConvNet for sentence identity decoding. Finally, our Grad-CAM visualization analysis identifies key areas influencing the network's predictions, offering valuable insights into the neural processes underlying language perception and comprehension. Significance. These findings advance our understanding of EEG-based speech perception decoding and hold promise for the development of speech neuroprostheses, particularly in scenarios where subjects cannot provide their own training data.

GCN-LSTM for EEG Classification Based on Unspoken Speech of Bilinguals.

EEG Recognition Based on Multiple Types of Information by Using Wavelet Packet Transform and Neural Networks

Silent EEG Classification Using Cross-Fusion Adaptive Graph Convolution Network for Multilingual Neurolinguistic Signal Decoding.

Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network

Characterizing Neural Entrainment to Hierarchical Linguistic Units using Electroencephalography (EEG).

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

An Explainable and Generalizable Recurrent Neural Network Approach for Differentiating Human Brain States on EEG Dataset

Decoding Silent Reading EEG Signals Using Adaptive Feature Graph Convolutional Network

Identification of perceived sentences using deep neural networks in EEG

Towards Linguistic Neural Representation Learning and Sentence Retrieval from Electroencephalogram Recordings

Electroencephalogram-Based Subject Matching Learning (ESML): A Deep Learning Framework on Electroencephalogram-Based Biometrics and Task Identification.

Decoding EEG in Motor Imagery Tasks with Graph Semi-Supervised Broad Learning

Classify EEG and Reveal Latent Graph Structure with Spatio-Temporal Graph Convolutional Neural Network

GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals

A Spiking Neural Network With Adaptive Graph Convolution and LSTM for EEG-Based Brain-Computer Interfaces

Continuous Silent Speech Recognition using EEG

Recognizing Tonal and Nontonal Mandarin Sentences for EEG-Based Brain–Computer Interface

EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretation

An Intelligent EEG Classification Methodology Based on Sparse Representation Enhanced Deep Learning Networks

Recognizing Tonal and Non-Tonal Mandarin Sentences for EEG-based Brain-Computer Interface

Classification of hand movements from EEG using a FusionNet based LSTM network