Abstract:Abstract Objective . Motor imagery (MI) electroencephalography (EEG) classification is regarded as a promising technology for brain–computer interface (BCI) systems, which help people to communicate with the outside world using neural activities. However, decoding human intent accurately is a challenging task because of its small signal-to-noise ratio and non-stationary characteristics. Methods that directly extract features from raw EEG signals ignores key frequency domain information. One of the challenges in MI classification tasks is finding a way to supplement the frequency domain information ignored by the raw EEG signal. Approach . In this study, we fuse different models using their complementary characteristics to develop a multiscale space-time-frequency feature-guided multitask learning convolutional neural network (CNN) architecture. The proposed method consists of four modules: the space-time feature-based representation module, time-frequency feature-based representation module, multimodal fused feature-guided generation module, and classification module. The proposed framework is based on multitask learning. The four modules are trained using three tasks simultaneously and jointly optimized. Results . The proposed method is evaluated using three public challenge datasets. Through quantitative analysis, we demonstrate that our proposed method outperforms most state-of-the-art machine learning and deep learning techniques for EEG classification, thereby demonstrating the robustness and effectiveness of our method. Moreover, the proposed method is employed to realize control of robot based on EEG signal, verifying its feasibility in real-time applications. Significance . To the best of our knowledge, a deep CNN architecture that fuses different input cases, which have complementary characteristics, has not been applied to BCI tasks. Because of the interaction of the three tasks in the multitask learning architecture, our method can improve the generalization and accuracy of subject-dependent and subject-independent methods with limited annotated data.

A Multi-class Graph Convolutional Neural Network for EEG Classification and Representation

A brain topography graph embedded convolutional neural network for EEG-based motor imagery classification

Efficient Multi-View Graph Convolutional Network with Self-Attention for Multi-Class Motor Imagery Decoding

Multi-Scale Convolutional Attention and Riemannian Geometry Network for EEG-Based Motor Imagery Classification

MST-DGCN: A Multi-Scale Spatio-Temporal and Dynamic Graph Convolution Fusion Network for Electroencephalogram Recognition of Motor Imagery

GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals

Graph neural network based on brain inspired forward-forward mechanism for motor imagery classification in brain-computer interfaces

A Compact Multi-Branch 1D Convolutional Neural Network for EEG-based Motor Imagery Classification

EEG-GNN: Graph Neural Networks for Classification of Electroencephalogram (EEG) Signals

Multi-Scale Spatiotemporal Attention Network for Neuron based Motor Imagery EEG Classification

Electroencephalogram-Based Motor Imagery Signals Classification Using a Multi-Branch Convolutional Neural Network Model with Attention Blocks

Motor Imagery Classification Based on Deep Convolutional Neural Network and Its Application in Exoskeleton Controlled by EEG

Multiattention Adaptation Network for Motor Imagery Recognition

EEG temporal information-based 1-D convolutional neural network for motor imagery classification

Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification

Multi-branch spatial-temporal-spectral convolutional neural networks for multi-task motor imagery EEG classification

A Multibranch of Convolutional Neural Network Models for Electroencephalogram-Based Motor Imagery Classification

A classification method for EEG motor imagery signals based on parallel convolutional neural network

A novel multi-branch hybrid neural network for motor imagery EEG signal classification

ETCNet: An EEG-based motor imagery classification model combining efficient channel attention and temporal convolutional network

Physics-Informed Attention Temporal Convolutional Network for EEG-Based Motor Imagery Classification