Abstract:The decoding of motor imagery (MI) electroencephalogram (EEG) is an essential component of the brain–computer interface (BCI), which can help patients with motor impairment communicate directly with the outside world through assistive devices. The key to motor imagery electroencephalogram (MI-EEG) classification is to extract multiple temporal, spatial, and spectral features, to obtain more comprehensive and representative information. However, current deep learning methods must fully consider the depth of temporal features and multi-spectral knowledge in EEG and often ignore the temporal or spectrum dependence in MI-EEG. In addition, the lack of effective feature fusion methods can lead to information redundancy, which affects decoding performance. To solve the above problems, this paper proposes a novel MI-EEG decoding method, named multi-scale feature fusion network based on attention mechanism (MSFF-SENet). Firstly, the multi-scale spatio-temporal module (MS-STM) and the multi-scale temporal module (MSTM) extract spatial and high-dimensional temporal features from the original signal. Then, the power spectral density estimation (PSD) convolution module (PSD-Conv module) acquires the multi-spectral features of the MI-EEG signal. Secondly, the feature fusion module fuses spatio-temporal and multi-spectral features to generate integrated feature mappings and establish the dependencies between different features. Finally, we conducted a visual analysis of the results, explaining the neural activity patterns of various motor imagery tasks in different frequency ranges and revealing the potential relationship between body movement and changes related to brain activity. The experimental results show that the classification accuracy of this model in BCI Competition IV 2a (BCI IV 2a) and High Gamma (HGD) datasets is 85.37% and 96.60%, respectively, which is superior to the most advanced methods.

Multiple Feature Domains Information Fusion for Computer-Aided Clinical Electromyography

Neural networks combination by fuzzy integral in clinical electromyography

A Hybrid Neuro-Fuzzy System for Neuromuscular Disorders Diagnosis

Automatic diagnostics of electroencephalography pathology based on multi-domain feature fusion

Motor intent recognition of multi-feature fusion EEG signals by UMAP algorithm

A Multi-Feature Fusion Decoding Study for Unilateral Upper-Limb Fine Motor Imagery

Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals

sEMG Pattern Recognition Based on Multi Feature Fusion of Wavelet Transform

Simultaneous EEG-fNIRS Data Classification Through Selective Channel Representation and Spectrogram Imaging

Efficient Feature Learning Model of Motor Imagery EEG Signals with L1-Norm and Weighted Fusion

A Multiscale Feature Fusion Network Based on Attention Mechanism for Motor Imagery EEG Decoding

Motor Imagery EEG Signal Classification Using Distinctive Feature Fusion with Adaptive Structural LASSO

Persistent Feature Analysis of Multimodal Brain Networks Using Generalized Fused Lasso for EMCI Identification

Canonical Feature Selection for Joint Regression and Multi-Class Identification in Alzheimer’s Disease Diagnosis

Multi-Scale and Multi-Channel Information Fusion for Exercise Electrocardiogram Feature Extraction and Classification

Surface Myoelectric Signal Information Fusion and Motion Classification

Domain Contrast Network for Cross-Muscle ALS Disease Identification with EMG Signal

Spatial and Spectral Features Fusion for EEG Classification during Motor Imagery in BCI

A Hybrid EEG-fNIRS Brain-Computer Interface Based on Dynamic Functional Connectivity and Long Short-Term Memory

Fusing Medical Image Features and Clinical Features with Deep Learning for Computer-Aided Diagnosis

Aided Diagnosis Model Based on Deep Learning for Glioblastoma, Solitary Brain Metastases, and Primary Central Nervous System Lymphoma with Multi-Modal MRI