Abstract:Abstract Context . Electroencephalography (EEG) is a complex signal and can require several years of training, as well as advanced signal processing and feature extraction methodologies to be correctly interpreted. Recently, deep learning (DL) has shown great promise in helping make sense of EEG signals due to its capacity to learn good feature representations from raw data. Whether DL truly presents advantages as compared to more traditional EEG processing approaches, however, remains an open question. Objective . In this work, we review 154 papers that apply DL to EEG, published between January 2010 and July 2018, and spanning different application domains such as epilepsy, sleep, brain–computer interfacing, and cognitive and affective monitoring. We extract trends and highlight interesting approaches from this large body of literature in order to inform future research and formulate recommendations. Methods . Major databases spanning the fields of science and engineering were queried to identify relevant studies published in scientific journals, conferences, and electronic preprint repositories. Various data items were extracted for each study pertaining to (1) the data, (2) the preprocessing methodology, (3) the DL design choices, (4) the results, and (5) the reproducibility of the experiments. These items were then analyzed one by one to uncover trends. Results . Our analysis reveals that the amount of EEG data used across studies varies from less than ten minutes to thousands of hours, while the number of samples seen during training by a network varies from a few dozens to several millions, depending on how epochs are extracted. Interestingly, we saw that more than half the studies used publicly available data and that there has also been a clear shift from intra-subject to inter-subject approaches over the last few years. About of the studies used convolutional neural networks (CNNs), while used recurrent neural networks (RNNs), most often with a total of 3–10 layers. Moreover, almost one-half of the studies trained their models on raw or preprocessed EEG time series. Finally, the median gain in accuracy of DL approaches over traditional baselines was across all relevant studies. More importantly, however, we noticed studies often suffer from poor reproducibility: a majority of papers would be hard or impossible to reproduce given the unavailability of their data and code. Significance . To help the community progress and share work more effectively, we provide a list of recommendations for future studies and emphasize the need for more reproducible research. We also make our summary table of DL and EEG papers available and invite authors of published work to contribute to it directly. A planned follow-up to this work will be an online public benchmarking portal listing reproducible results.

fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships

Beyond the Surface: Revealing the Depths of Brain Activity by Predicting fMRI from EEG with Deep Learning

Exploiting Asymmetric EEG Signals with EFD in Deep Learning Domain for Robust BCI

A New Generation of Brain-Computer Interfaces Driven by Discovery of Latent EEG-fMRI Linkages Using Tensor Decomposition

NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mapping

EEG to fMRI Synthesis: Is Deep Learning a candidate?

Interpretable Deep Neural Networks for Single-Trial EEG Classification

EEG to fMRI Synthesis Benefits from Attentional Graphs of Electrode Relationships

Towards best practice of interpreting deep learning models for EEG-based brain computer interfaces

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

Decoding fMRI brain states in real-time

Multimodal EEG-fMRI: Advancing insight into large-scale human brain dynamics

Pushing the Limits of EEG: Estimation of Large-Scale Functional Brain Networks and Their Dynamics Validated by Simultaneous fMRI

Beyond functional connectivity: deep learning applied to resting-state fMRI time series in the prediction of 58 human traits in the HCP

Deep learning-based electroencephalography analysis: a systematic review

A Realistic Framework for Investigating Decision Making in the Brain With High Spatiotemporal Resolution Using Simultaneous EEG/fMRI and Joint ICA

Towards Open-World EEG Decoding via Deep Learning

Brain Decoding Using Deep Convolutional Network and Its Application in Cross-Subject Analysis.

Deep learning in fNIRS: a review

A Hybrid Spatiotemporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multilevel Core Functional Components in Decoding Multitask Fmri Signals.