Abstract:Abstract Objective . Motor imagery (MI) electroencephalography (EEG) classification is regarded as a promising technology for brain–computer interface (BCI) systems, which help people to communicate with the outside world using neural activities. However, decoding human intent accurately is a challenging task because of its small signal-to-noise ratio and non-stationary characteristics. Methods that directly extract features from raw EEG signals ignores key frequency domain information. One of the challenges in MI classification tasks is finding a way to supplement the frequency domain information ignored by the raw EEG signal. Approach . In this study, we fuse different models using their complementary characteristics to develop a multiscale space-time-frequency feature-guided multitask learning convolutional neural network (CNN) architecture. The proposed method consists of four modules: the space-time feature-based representation module, time-frequency feature-based representation module, multimodal fused feature-guided generation module, and classification module. The proposed framework is based on multitask learning. The four modules are trained using three tasks simultaneously and jointly optimized. Results . The proposed method is evaluated using three public challenge datasets. Through quantitative analysis, we demonstrate that our proposed method outperforms most state-of-the-art machine learning and deep learning techniques for EEG classification, thereby demonstrating the robustness and effectiveness of our method. Moreover, the proposed method is employed to realize control of robot based on EEG signal, verifying its feasibility in real-time applications. Significance . To the best of our knowledge, a deep CNN architecture that fuses different input cases, which have complementary characteristics, has not been applied to BCI tasks. Because of the interaction of the three tasks in the multitask learning architecture, our method can improve the generalization and accuracy of subject-dependent and subject-independent methods with limited annotated data.

Joint Filter-Band-Combination and Multi-View CNN for Electroencephalogram Decoding.

A Multi-Domain Convolutional Neural Network for EEG-Based Motor Imagery Decoding

Attention-based convolutional neural network with multi-modal temporal information fusion for motor imagery EEG decoding

Motor Imagery Classification Based on Deep Convolutional Neural Network and Its Application in Exoskeleton Controlled by EEG

Motor Imagery EEG Decoding Based on Multi-scale Hybrid Networks and Feature Enhancement.

Exploiting Asymmetric EEG Signals with EFD in Deep Learning Domain for Robust BCI

FBMSNet: A Filter-Bank Multi-Scale Convolutional Neural Network for EEG-Based Motor Imagery Decoding

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification

Enhancing EEG Motor Imagery Decoding Performance Via Deep Temporal-domain Information Extraction

Joint Spatial and Temporal Features Extraction for Multi-Classification of Motor Imagery EEG.

Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification

Deep Convolutional Neural Network for EEG-Based Motor Decoding

A Channel-Projection Mixed-Scale Convolutional Neural Network for Motor Imagery EEG Decoding

A parallel multi-scale time-frequency block convolutional neural network based on channel attention module for motor imagery classification

A Simplified CNN Classification Method for MI-EEG via the Electrode Pairs Signals

Decoding Motor Imagery Tasks Using ESI and Hybrid Feature CNN

MI-MBFT: Superior Motor Imagery Decoding of Raw EEG Data Based on a Multi-Branch and Fusion Transformer Framework

Image-based Motor Imagery EEG Classification Using Convolutional Neural Network

One-Dimensional Convolutional Multi-branch Fusion Network for EEG-Based Motor Imagery Classification

A Model Combining Multi Branch Spectral-Temporal CNN, Efficient Channel Attention, and LightGBM for MI-BCI Classification

EISATC-Fusion: Inception Self-Attention Temporal Convolutional Network Fusion for Motor Imagery EEG Decoding