Abstract:Decoding brain activity related to specific tasks, such as imagining something, is important for brain computer interface (BCI) control. While decoding of brain signals, such as functional magnetic resonance imaging (fMRI) signals and electroencephalography (EEG) signals, during observing visual images and while imagining images has been previously reported, further development of methods for improving training, performance, and interpretation of brain data was the goal of this study. We applied a Sinc-EEGNet to decode brain activity during perception and imagination of visual stimuli, and added an attention module to extract the importance of each electrode or frequency band. We also reconstructed images from brain activity by using a generative adversarial network (GAN). By combining the EEG recorded during a visual task (perception) and an imagination task, we have successfully boosted the accuracy of classifying EEG data in the imagination task and improved the quality of reconstruction by GAN. Our result indicates that the brain activity evoked during the visual task is present in the imagination task and can be used for better classification of the imagined image. By using the attention module, we can derive the spatial weights in each frequency band and contrast spatial or frequency importance between tasks from our model. Imagination tasks are classified by low frequency EEG signals over temporal cortex, while perception tasks are classified by high frequency EEG signals over occipital and frontal cortex. Combining data sets in training results in a balanced model improving classification of the imagination task without significantly changing performance in the visual task. Our approach not only improves performance and interpretability but also potentially reduces the burden on training since we can improve the accuracy of classifying a relatively hard task with high variability (imagination) by combining with the data of the relatively easy task, observing visual images.

Image classification and reconstruction from low-density EEG

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion

Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion models

Image Reconstruction from Electroencephalography Using Latent Diffusion

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

Decoding Natural Images from EEG for Object Recognition

EEG-ImageNet: An Electroencephalogram Dataset and Benchmarks with Image Visual Stimuli of Multi-Granularity Labels

Decoding Brain Signals from Rapid-Event EEG for Visual Analysis Using Deep Learning

Reconstructing Visual Stimulus Images from EEG Signals Based on Deep Visual Representation Model

Improving classification and reconstruction of imagined images from EEG signals

EEG source imaging of hand movement-related areas: an evaluation of the reconstruction and classification accuracy with optimized channels

Brain decoding: toward real-time reconstruction of visual perception

Decoding Visual Recognition of Objects from EEG Signals based on Attention-Driven Convolutional Neural Network

EEG2IMAGE: Image Reconstruction from EEG Brain Signals

Applying advanced machine learning models to classify electro-physiological activity of human brain for use in biometric identification

EEG-based Image Feature Extraction for Visual Classification using Deep Learning

Neurodm: Decoding and Visualizing Human Brain Activity with Eeg-Guided Diffusion Model

EEG classification for visual brain decoding with spatio-temporal and transformer based paradigms

EEG-Based Visual Classification with Recurrent Neural Network

Classification of Visual Imagery and Imagined Speech EEG based Brain Computer Interfaces using 1D Convolutional Neural Network

Brain State Decoding for Rapid Image Retrieval