Abstract:In the Motor Imagery (MI)-based Brain Computer Interface (BCI), users' intention is converted into a control signal through processing a specific pattern in brain signals reflecting motor characteristics. There are such restrictions as the limited size of the existing datasets and low signal to noise ratio in the classification of MI Electroencephalogram (EEG) signals. Machine learning (ML) methods, particularly Deep Learning (DL), have overcome these limitations relatively. In this study, three hybrid models were proposed to classify the EEG signal in the MI-based BCI. The proposed hybrid models consist of the convolutional neural networks (CNN) and the Long-Short Term Memory (LSTM). In the first model, the CNN with different number of convolutional-pooling blocks (from shallow to deep CNN) was examined; a two-block CNN model not affected by the vanishing gradient descent and yet able to extract desirable features employed; the second and third models contained pre-trained CNNs conducing to the exploration of more complex features. The transfer learning strategy and data augmentation methods were applied to overcome the limited size of the datasets by transferring learning from one model to another. This was achieved by employing two powerful pre-trained convolutional neural networks namely ResNet-50 and Inception-v3. The continuous wavelet transform (CWT) was used to generate images for the CNN. The performance of the proposed models was evaluated on the BCI Competition IV dataset 2a. The mean accuracy vlaues of 86%, 90%, and 92%, and mean Kappa values of 81%, 86%, and 88% were obtained for the hybrid neural network with the customized CNN, the hybrid neural network with ResNet-50 and the hybrid neural network with Inception-v3, respectively. Despite the promising performance of the three proposed models, the hybrid neural network with Inception-v3 outperformed the two other models. The best obtained result in the present study improved the previous best result in the literature by 7% in terms of classification accuracy. From the findings, it can be concluded that transfer learning based on a pre-trained CNN in combination with LSTM is a novel method in MI-based BCI. The study also has implications for the discrimination of motor imagery tasks in each EEG recording channel and in different brain regions which can reduce computational time in future works by only selecting the most effective channels.

Motor imagery EEG classification algorithm based on CNN-LSTM feature fusion network

Motor Imagery Classification Based on Deep Convolutional Neural Network and Its Application in Exoskeleton Controlled by EEG

Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification

A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals

Parallel Spatial–Temporal Self-Attention CNN-Based Motor Imagery Classification for BCI

An end-to-end multi-task motor imagery EEG classification neural network based on dynamic fusion of spectral-temporal features

Multiscale Spatial-Temporal Feature Fusion Neural Network for Motor Imagery Brain-Computer Interfaces

Classification algorithm for EEG-based motor imagery using hybrid neural network with spatio-temporal convolution and multi-head attention mechanism

Bidirectional feature pyramid attention-based temporal convolutional network model for motor imagery electroencephalogram classification

ETCNet: An EEG-based motor imagery classification model combining efficient channel attention and temporal convolutional network

3D-CLMI: A Motor Imagery EEG Classification Model via Fusion of 3D-CNN and LSTM with Attention

One-Dimensional Convolutional Multi-branch Fusion Network for EEG-Based Motor Imagery Classification

A parallel multi-scale time-frequency block convolutional neural network based on channel attention module for motor imagery classification

A novel multi-branch hybrid neural network for motor imagery EEG signal classification

Self-attention-based Convolutional Neural Network AND Time-frequency Common Spatial Pattern for enhanced Motor Imagery Classification

Classification algorithm for motor imagery fusing CNN and attentional mechanisms based on functional near-infrared spectroscopy brain image

Short time Fourier transformation and deep neural networks for motor imagery brain computer interface recognition.

Single-trial EEG Classification of Motor Imagery Using Deep Convolutional Neural Networks

EEG-FMCNN: A fusion multi-branch 1D convolutional neural network for EEG-based motor imagery classification

EISATC-Fusion: Inception Self-Attention Temporal Convolutional Network Fusion for Motor Imagery EEG Decoding

3D Convolution neural network with multiscale spatial and temporal cues for motor imagery EEG classification