3D-CLMI: A Motor Imagery EEG Classification Model via Fusion of 3D-CNN and LSTM with Attention

Shiwei Cheng,Yuejiang Hao

2023-12-20

Abstract:Due to the limitations in the accuracy and robustness of current electroencephalogram (EEG) classification algorithms, applying motor imagery (MI) for practical Brain-Computer Interface (BCI) applications remains challenging. This paper proposed a model that combined a three-dimensional convolutional neural network (CNN) with a long short-term memory (LSTM) network with attention to classify MI-EEG signals. This model combined MI-EEG signals from different channels into three-dimensional features and extracted spatial features through convolution operations with multiple three-dimensional convolutional kernels of different scales. At the same time, to ensure the integrity of the extracted MI-EEG signal temporal features, the LSTM network was directly trained on the preprocessed raw signal. Finally, the features obtained from these two networks were combined and used for classification. Experimental results showed that this model achieved a classification accuracy of 92.7% and an F1-score of 0.91 on the public dataset BCI Competition IV dataset 2a, which were both higher than the state-of-the-art models in the field of MI tasks. Additionally, 12 participants were invited to complete a four-class MI task in our lab, and experiments on the collected dataset showed that the 3D-CLMI model also maintained the highest classification accuracy and F1-score. The model greatly improved the classification accuracy of users' motor imagery intentions, giving brain-computer interfaces better application prospects in emerging fields such as autonomous vehicles and medical rehabilitation.

Human-Computer Interaction,Machine Learning,Signal Processing

What problem does this paper attempt to address?

This paper aims to address the limitations of current electroencephalogram (EEG) classification algorithms in terms of accuracy and robustness, thereby improving the performance of motor imagery (MI) tasks in brain-computer interface (BCI) applications. Specifically, the paper proposes a new model that combines a three-dimensional convolutional neural network (3D-CNN) and a long short-term memory network (LSTM) with an attention mechanism (referred to as 3D-CLMI) for classifying MI-EEG signals. The main contributions of this model are as follows: 1. **Spatial Feature Extraction**: Extracting spatial features between different channels through multi-scale three-dimensional convolutional kernels. 2. **Temporal Feature Preservation**: Training the LSTM network directly on the preprocessed raw signals to ensure the integrity of temporal features. 3. **Feature Fusion and Classification**: Combining the features extracted by the two networks for final classification. Experimental results show that the model achieves a classification accuracy of 92.7% and an F1 score of 0.91 on the publicly available BCI Competition IV dataset 2a, outperforming existing methods. Additionally, on datasets collected in the laboratory, the model also demonstrates the best classification accuracy and F1 score, significantly improving the classification accuracy of users' motor imagery intentions. This provides better prospects for the application of brain-computer interfaces in emerging fields such as autonomous driving vehicles and medical rehabilitation.

3D-CLMI: A Motor Imagery EEG Classification Model via Fusion of 3D-CNN and LSTM with Attention

Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network

Motor Imagery Classification Based on Deep Convolutional Neural Network and Its Application in Exoskeleton Controlled by EEG

Classification of Motor Imagery Based on Multi-Scale Feature Extraction and the Channel-Temporal Attention Module.

3D Convolution neural network with multiscale spatial and temporal cues for motor imagery EEG classification

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification

ETCNet: An EEG-based motor imagery classification model combining efficient channel attention and temporal convolutional network

Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification

One-Dimensional Convolutional Multi-branch Fusion Network for EEG-Based Motor Imagery Classification

Temporal-frequency-phase feature classification using 3D-convolutional neural networks for motor imagery and movement

Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI

A Densely Connected Multi-Branch 3D Convolutional Neural Network for Motor Imagery EEG Decoding

A spatial-frequency-temporal 3D convolutional neural network for motor imagery EEG signal classification

An effective classification approach for EEG-based motor imagery tasks combined with attention mechanisms

A motor imagery classification model based on hybrid brain-computer interface and multitask learning of electroencephalographic and electromyographic deep features

A Multi-Branch 3D Convolutional Neural Network for EEG-Based Motor Imagery Classification.

Multiclass classification of motor imagery tasks based on multi-branch convolutional neural network and temporal convolutional network model

A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals

Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification

Classification algorithm for EEG-based motor imagery using hybrid neural network with spatio-temporal convolution and multi-head attention mechanism

A new attention-based 3D densely connected cross-stage-partial network for motor imagery classification in BCI