Abstract:Abstract Objective . Motor imagery (MI) electroencephalography (EEG) classification is regarded as a promising technology for brain–computer interface (BCI) systems, which help people to communicate with the outside world using neural activities. However, decoding human intent accurately is a challenging task because of its small signal-to-noise ratio and non-stationary characteristics. Methods that directly extract features from raw EEG signals ignores key frequency domain information. One of the challenges in MI classification tasks is finding a way to supplement the frequency domain information ignored by the raw EEG signal. Approach . In this study, we fuse different models using their complementary characteristics to develop a multiscale space-time-frequency feature-guided multitask learning convolutional neural network (CNN) architecture. The proposed method consists of four modules: the space-time feature-based representation module, time-frequency feature-based representation module, multimodal fused feature-guided generation module, and classification module. The proposed framework is based on multitask learning. The four modules are trained using three tasks simultaneously and jointly optimized. Results . The proposed method is evaluated using three public challenge datasets. Through quantitative analysis, we demonstrate that our proposed method outperforms most state-of-the-art machine learning and deep learning techniques for EEG classification, thereby demonstrating the robustness and effectiveness of our method. Moreover, the proposed method is employed to realize control of robot based on EEG signal, verifying its feasibility in real-time applications. Significance . To the best of our knowledge, a deep CNN architecture that fuses different input cases, which have complementary characteristics, has not been applied to BCI tasks. Because of the interaction of the three tasks in the multitask learning architecture, our method can improve the generalization and accuracy of subject-dependent and subject-independent methods with limited annotated data.

Classification of imagined speech of vowels from EEG signals using multi-headed CNNs feature fusion network

EEG Recognition Based on Multiple Types of Information by Using Wavelet Packet Transform and Neural Networks

Resting state EEG assisted imagined vowel phonemes recognition by native and non-native speakers using brain connectivity measures

Towards Imagined Speech: Identification of Brain States from EEG Signals for BCI-based Communication Systems

Recognition of EEG Signals from Imagined Vowels Using Deep Learning Methods

Imagined Speech Classification Using EEG and Deep Learning

Inner Speech Classification using EEG Signals: A Deep Learning Approach

Exploiting Asymmetric EEG Signals with EFD in Deep Learning Domain for Robust BCI

Deep Learning the EEG Manifold for Phonological Categorization from Active Thoughts

Identification of vowels in consonant–vowel–consonant words from speech imagery based EEG signals

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

EEG Classification of Motor Imagery Using a Novel Deep Learning Framework

Inferring imagined speech using EEG signals: a new approach using Riemannian manifold features

Classification of Visual Imagery and Imagined Speech EEG based Brain Computer Interfaces using 1D Convolutional Neural Network

The design and implementation of multi-character classification scheme based on EEG signals of visual imagery

Imagined speech classification exploiting EEG power spectrum features

Learning Patterns in Imaginary Vowels for an Intelligent Brain Computer Interface (BCI) Design

EEG classification using a simple CNN model for imagined and executed motor signals

Decoding Imagined Speech and Computer Control using Brain Waves

Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification

Imagined Speech State Classification for Robust Brain-Computer Interface