Abstract:Recent advances in deep learning have had a methodological and practical impact on brain-computer interface (BCI) research. Among the various deep network architectures, convolutional neural networks (CNNs) have been well suited for spatio-spectral-temporal electroencephalogram (EEG) signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in different ranges of frequency components. To better reflect these multi-frequency properties in EEGs, we propose a novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts relationships among electrodes, i.e., spatial representations, for subject intention/condition identification. Furthermore, by completely representing EEG signals with spatio-spectral-temporal information, the proposed method can be utilized for diverse paradigms in both active and passive BCIs, contrary to existing methods that are primarily focused on single-paradigm BCIs. To demonstrate the validity of our proposed method, we conducted experiments on various paradigms of active/passive BCI datasets. Our experimental results demonstrated that the proposed method achieved performance improvements when judged against comparable state-of-the-art methods. Additionally, we analyzed the proposed method using different techniques, such as PSD curves and relevance score inspection to validate the multi-scale EEG signal information capturing ability, activation pattern maps for investigating the learned spatial filters, and t-SNE plotting for visualizing represented features. Finally, we also demonstrated our method's application to real-world problems. Based on our experimental results and analyses, we believe that the proposed-multi-scale neural network can be useful for various BCI paradigms, as a starting model or as a backbone network in any new BCI experiments.

Electroencephalographic (EEG) control of cursor movement in three-dimensional scene based on Small-World neural network

Improving NeuCube Spiking Neural Network for EEG-based Pattern Recognition Using Transfer Learning

EEG-based cursor control system

A Combined Projection for Remote Control of a Vehicle Based on Movement Imagination: A Single Trial Brain Computer Interface Study

Multi-Scale Neural Network for EEG Representation Learning in BCI

A Method for the Spatial Interpolation of EEG Signals Based on the Bidirectional Long Short-Term Memory Network

Single-Trial EEG Classification Using Spatio-Temporal Weighting and Correlation Analysis for RSVP-Based Collaborative Brain Computer Interface.

Navigation Learning Assessment Using EEG-Based Multi-Time Scale Spatiotemporal Compound Model

EEG Signal Processing Based on Multivariate Empirical Mode Decomposition and Common Spatial Pattern Hybrid Algorithm

Towards Real-World BCI: CCSPNet, A Compact Subject-Independent Motor Imagery Framework

Multi-channel EEG signal feature extraction and pattern recognition on horizontal mental imagination task of 1-D cursor movement for brain computer interface

A novel brain-computer interface based on audio-assisted visual evoked EEG and spatial-temporal attention CNN

Task-State EEG Signal Classification for Spatial Cognitive Evaluation Based on Multiscale High-Density Convolutional Neural Network

Conversion of EEG activity into cursor movement by a brain-computer interface (BCI)

A Robust 3D-Convolutional Neural Network-Based Electroencephalogram Decoding Model for the Intra-Individual Difference

A Spiking Neural Network With Adaptive Graph Convolution and LSTM for EEG-Based Brain-Computer Interfaces

Electrophysiological Brain Network Estimation with Simultaneous Scalp EEG and Intracranial EEG: Inference Algorithm and Applications

A Three-Dimensional Spatio-Temporal Eeg Pattern Analyzing System

Imagined hand movement identification based on spatio-temporal pattern recognition of EEG

Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI

Learning EEG topographical representation for classification via convolutional neural network