Abstract:Objective. Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain–computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks? Approach. A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture. Main results. For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review. Significance. This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.Electroencephalography (EEG) is widely used in research involving neural engineering, neuroscience, and biomedical engineering (e.g. brain computer interfaces, BCI) [<a href="#jneab0ab5bib001">1</a>]; sleep analysis [<a href="#jneab0ab5bib002">2</a>]; and seizure detection [<a href="#jneab0ab5bib003">3</a>]) because of its high temporal resolution, non-invasiveness, and relatively low financial cost. The automatic classification of these signals is an important step towards making the use of EEG more practical in application and less reliant on trained professionals. The typical EEG classification pipeline includes artifact removal, feature extraction, and classification. On the most basic level, an EEG dataset consists of a 2D (time and channel) matrix of real values that represent brain-generated potentials recorded on the scalp associated with specific task conditions [<a href="#jneab0ab5bib004">4</a>]. This highly structured form makes EEG data suitable for machine learning. A great number of traditional machine learning and pattern recognition algorithms have been applied on the EEG data. For example, independent component analysis (ICA) is commonly used for artifact removal [<a href="#jneab0ab5bib005">5</a>]; principle component analysis (PCA) and local Fisher's discriminant analysis (LFDA) are typically used to reduce dimensionality of the features [<a href="#jneab0ab5bib005">5</a>]; classic supervised learning methods such as linear discriminant analysis (LDA), support vector machines (SVM), and decision trees are common in neural classification [<a href="#jneab0ab5bib006">6</a>, <a href="#jneab0ab5bib007">7</a>]; and canonical correlation analysis (CCA) is frequently used to identify steady-state visual evoked potentials (SSVEPs).Neural networks did not immediately receive the high attention seen today in neural classification applications because of practical issues, such as very long computation time and problems with the vanishing/exploding gradients [<a href="#jneab0ab5bib008">8</a>]. Fortunately, the availability of large datasets and the recent development of graphic processing units (GPU's) brought neural network researchers an inexpensive and powerful solution to their hardware bottleneck [<a href="#jneab0ab5bib009">9</a>], allowing them to investigate deep learning architectures (neural network architectures containing at least two hidden layers). These innovations have led to an exponential increase in interest and applications of -Abstract Truncated-

Relating EEG to continuous speech using deep neural networks: a review

[The Neural Encoding of Continuous Speech - Recent Advances in EEG and MEG Studies].

Deep learning-based electroencephalography analysis: a systematic review

Continuous and discrete decoding of overt speech with scalp electroencephalography (EEG)

Decoding Envelope and Frequency-Following EEG Responses to Continuous Speech Using Deep Neural Networks

Continuous and discrete decoding of overt speech with electroencephalography

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

Relating the fundamental frequency of speech with EEG using a dilated convolutional network

Deep learning for electroencephalogram (EEG) classification tasks: a review

Deep Learning for Micro-Electrocorticographic (µecog) Data

Continuous Silent Speech Recognition using EEG

Identification of perceived sentences using deep neural networks in EEG

Deep Learning in EEG: Advance of the Last Ten-Year Critical Period

Study on neural entrainment to continuous speech using dynamic source connectivity analysis

Deep learning for EEG data analytics: A survey

Linear versus deep learning methods for noisy speech separation for EEG-informed attention decoding

Comprehensive Review of EEG-to-Output Research: Decoding Neural Signals into Images, Videos, and Audio

Deep Representation Learning for Open Vocabulary Electroencephalography-to-Text Decoding

Classifying native versus foreign speech perception from EEG using linguistic speech features

Inner Speech Classification using EEG Signals: A Deep Learning Approach

Speech decoding from stereo-electroencephalography (sEEG) signals using advanced deep learning methods