Abstract:Objetive. Decoding speech from brain activity can enable communication for individuals with speech disorders. Deep neural networks have shown great potential for speech decoding applications. However, the limited availability of large datasets containing neural recordings from speech-impaired subjects poses a challenge. Leveraging data from healthy participants can mitigate this limitation and expedite the development of speech neuroprostheses while minimizing the need for patient-specific training data. Approach. In this study, we collected a substantial dataset consisting of recordings from 56 healthy participants using 64 EEG channels. Multiple neural networks were trained to classify perceived sentences in the Spanish language using subject-independent, mixed-subjects, and fine-tuning approaches. The dataset has been made publicly available to foster further research in this area. Main results. Our results demonstrate a remarkable level of accuracy in distinguishing sentence identity across 30 classes, showcasing the feasibility of training Deep Neural Networks (DNNs) to decode sentence identity from perceived speech using EEG. Notably, the subject-independent approach rendered accuracy comparable to the mixed-subjects approach, although with higher variability among subjects. Additionally, our fine-tuning approach yielded even higher accuracy, indicating an improved capability to adapt to individual subject characteristics, which enhances performance. This suggests that DNNs have effectively learned to decode universal features of brain activity across individuals while also being adaptable to specific participant data. Furthermore, our analyses indicate that EEGNet and DeepConvNet exhibit comparable performance, outperforming ShallowConvNet for sentence identity decoding. Finally, our Grad-CAM visualization analysis identifies key areas influencing the network's predictions, offering valuable insights into the neural processes underlying language perception and comprehension. Significance. These findings advance our understanding of EEG-based speech perception decoding and hold promise for the development of speech neuroprostheses, particularly in scenarios where subjects cannot provide their own training data.

Deep learning with convolutional neural networks for decoding and visualization of EEG pathology

Deep learning with convolutional neural networks for EEG decoding and visualization

Deep Learning for Micro-Electrocorticographic (µecog) Data

DeepECG: Image-based Electrocardiogram Interpretation with Deep Convolutional Neural Networks

Residual Deep Convolutional Neural Network for EEG Signal Classification in Epilepsy

Reaching the ceiling? Empirical scaling behaviour for deep EEG pathology classification

The Abnormal Detection of Electroencephalogram With Three-Dimensional Deep Convolutional Neural Networks

Epileptic Seizure Detection with Deep EEG Features by Convolutional Neural Network and Shallow Classifiers

Deep Riemannian Networks for EEG Decoding

Deep Classification of Epileptic Signals

Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals

Deep Invertible Networks for EEG-based brain-signal decoding

Decoding Silent Reading EEG Signals Using Adaptive Feature Graph Convolutional Network

Deep Learning with Convolutional Neural Networks for Motor Brain-Computer Interfaces Based on Stereo-Electroencephalography (SEEG)

Generalized neural decoders for transfer learning across participants and recording modalities

Deep learning for EEG data analytics: A survey

Deep-EEG: An Optimized and Robust Framework and Method for EEG-Based Diagnosis of Epileptic Seizure

Identification of perceived sentences using deep neural networks in EEG

Cross-paradigm pretraining of convolutional networks improves intracranial EEG decoding

Speech decoding from stereo-electroencephalography (sEEG) signals using advanced deep learning methods

Automatic Analysis of EEGs Using Big Data and Hybrid Deep Learning Architectures