Abstract:Background: Speech decoding, one of the most intriguing BCI applications, opens up plentiful opportunities from rehabilitation of patients to direct and seamless communication between human species. Typical solutions rely on invasive recordings with a large number of distributed electrodes implanted through craniotomy. Here we explored the possibility of creating speech prosthesis in a minimally invasive setting with a small number of spatially segregated intracranial electrodes. Methods: We collected one hour of data (from two sessions) in two patients implanted with invasive electrodes. We then used only the contacts that pertained to a single sEEG shaft or an ECoG stripe to decode neural activity into 26 words and one silence class. We employed a compact convolutional network-based architecture whose spatial and temporal filter weights allow for a physiologically plausible interpretation. Results: We achieved on average 55\% accuracy using only 6 channels of data recorded with a single minimally invasive sEEG electrode in the first patient and 70\% accuracy using only 8 channels of data recorded for a single ECoG strip in the second patient in classifying 26+1 overtly pronounced words. Our compact architecture did not require the use of pre-engineered features, learned fast and resulted in a stable, interpretable and physiologically meaningful decision rule successfully operating over a contiguous dataset collected during a different time interval than that used for training. Spatial characteristics of the pivotal neuronal populations corroborate with active and passive speech mapping results and exhibit the inverse space-frequency relationship characteristic of neural activity. Comparison to other architectures our compact solution performed on par or better than those recently featured in neural speech decoding literature. Conclusions: We showcase the possibility of building speech prosthesis with a small number of electrodes and based on a compact feature engineering free decoder derived from a small amount of training data.

Speech synthesis from neural decoding of spoken sentences

A high-performance neuroprosthesis for speech decoding and avatar control

A neural speech decoding framework leveraging deep learning and speech synthesis

Real-time Synthesis of Imagined Speech Processes from Minimally Invasive Recordings of Neural Activity

The speech neuroprosthesis

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework

Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities

Towards Naturalistic Speech Decoding from Intracranial Brain Data

Natural speech re-synthesis from direct cortical recordings using a pre-trained encoder-decoder framework

A wireless brain-machine interface for real-time speech synthesis

An instantaneous voice synthesis neuroprosthesis

A high-performance speech neuroprosthesis

A bilingual speech neuroprosthesis driven by cortical articulatory representations shared between languages

High-resolution neural recordings improve the accuracy of speech decoding

A Speech Neuroprosthesis in the Frontal Lobe and Hippocampus: Decoding High-Frequency Activity into Phonemes

Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network

An accurate and rapidly calibrating speech neuroprosthesis

NeuSpeech: Decode Neural signal as Speech

Decoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals