Abstract:Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship. In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (<a class="link-external link-http" href="http://ClinicalTrials.gov" rel="external noopener nofollow">this http URL</a> identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron (MLP), convolutional neural networks (CNN), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity. Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.

Advantages of Deep Learning for ECoG-based Speech Recognition

Deep Learning for Micro-Electrocorticographic (µecog) Data

Progress in speech decoding from the electrocorticogram

Fast and Accurate Decoding of Finger Movements from ECoG Through Riemannian Features and Modern Machine Learning Techniques

Enhanced Classification of Individual Finger Movements with ECoG.

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

High-resolution neural recordings improve the accuracy of speech decoding

Speech decoding from stereo-electroencephalography (sEEG) signals using advanced deep learning methods

Rapid Decoding of Hand Gestures in Electrocorticography Using Recurrent Neural Networks.

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

Brain Signals to Rescue Aphasia, Apraxia and Dysarthria Speech Recognition

A neural speech decoding framework leveraging deep learning and speech synthesis

Continuous and discrete decoding of overt speech with electroencephalography

The Potential for a Speech Brain–Computer Interface Using Chronic Electrocorticography

Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework

Decoding ECoG signal into 3D hand translation using deep learning

Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals

Real-time detection of spoken speech from unlabeled ECoG signals: A pilot study with an ALS participant

An ECoG-Based BCI Based on Auditory Attention to Natural Speech

Brain2Char: A Deep Architecture for Decoding Text from Brain Recordings

Speech decoding using cortical and subcortical electrophysiological signals