Abstract:Background: The adoption of convolutional neural networks (CNNs) for decoding electroencephalogram (EEG)-based motor imagery (MI) in brain-computer interfaces has significantly increased recently. The effective extraction of motor imagery features is vital due to the variability among individuals and temporal states. Methods: This study introduces a novel network architecture, 3D-convolutional neural network-generative adversarial network (3D-CNN-GAN), for decoding both within-session and cross-session motor imagery. Initially, EEG signals were extracted over various time intervals using a sliding window technique, capturing temporal, frequency, and phase features to construct a temporal-frequency-phase feature (TFPF) three-dimensional feature map. Generative adversarial networks (GANs) were then employed to synthesize artificial data, which, when combined with the original datasets, expanded the data capacity and enhanced functional connectivity. Moreover, GANs proved capable of learning and amplifying the brain connectivity patterns present in the existing data, generating more distinctive brain network features. A compact, two-layer 3D-CNN model was subsequently developed to efficiently decode these TFPF features. Results: Taking into account session and individual differences in EEG data, tests were conducted on both the public GigaDB dataset and the SHU laboratory dataset. On the GigaDB dataset, our 3D-CNN and 3D-CNN-GAN models achieved two-class within-session motor imagery accuracies of 76.49% and 77.03%, respectively, demonstrating the algorithm's effectiveness and the improvement provided by data augmentation. Furthermore, on the SHU dataset, the 3D-CNN and 3D-CNN-GAN models yielded two-class within-session motor imagery accuracies of 67.64% and 71.63%, and cross-session motor imagery accuracies of 58.06% and 63.04%, respectively. Conclusions: The 3D-CNN-GAN algorithm significantly enhances the generalizability of EEG-based motor imagery brain-computer interfaces (BCIs). Additionally, this research offers valuable insights into the potential applications of motor imagery BCIs.

MindCeive: Perceiving human imagination using CNN-GRU and GANs

Neurocognition-inspired Design with Machine Learning

Mind Artist: Creating Artistic Snapshots with Human Thought

NeuroGAN: image reconstruction from EEG signals via an attention-based GAN

Improving classification and reconstruction of imagined images from EEG signals

MindGPT: Interpreting What You See with Non-invasive Brain Recordings

A GAN model encoded by CapsEEGNet for visual EEG encoding and image reproduction

A General Framework for Revealing Human Mind with auto-encoding GANs

EEG2IMAGE: Image Reconstruction from EEG Brain Signals

Decoding human brain activity with deep learning

BrainDreamer: Reasoning-Coherent and Controllable Image Generation from EEG Brain Signals via Language Guidance

Human-in-the-loop design with machine learning

Salient Arithmetic Data Extraction from Brain Activity via an Improved Deep Network

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

A Visual Characteristic Land-Scape Design for EEG Signal Based on LSTM-GAN

Visual Saliency and Image Reconstruction from EEG Signals via an Effective Geometric Deep Network-Based Generative Adversarial Network

Deep Learning Human Mind for Automated Visual Classification

Brain Captioning: Decoding human brain activity into images and text

Cross-Subject EEG Feedback for Implicit Image Generation

Verifying Design through Generative Visualization of Neural Activities

EEG-Based Feature Classification Combining 3D-Convolutional Neural Networks with Generative Adversarial Networks for Motor Imagery