Abstract:Decoding brain activity related to specific tasks, such as imagining something, is important for brain computer interface (BCI) control. While decoding of brain signals, such as functional magnetic resonance imaging (fMRI) signals and electroencephalography (EEG) signals, during observing visual images and while imagining images has been previously reported, further development of methods for improving training, performance, and interpretation of brain data was the goal of this study. We applied a Sinc-EEGNet to decode brain activity during perception and imagination of visual stimuli, and added an attention module to extract the importance of each electrode or frequency band. We also reconstructed images from brain activity by using a generative adversarial network (GAN). By combining the EEG recorded during a visual task (perception) and an imagination task, we have successfully boosted the accuracy of classifying EEG data in the imagination task and improved the quality of reconstruction by GAN. Our result indicates that the brain activity evoked during the visual task is present in the imagination task and can be used for better classification of the imagined image. By using the attention module, we can derive the spatial weights in each frequency band and contrast spatial or frequency importance between tasks from our model. Imagination tasks are classified by low frequency EEG signals over temporal cortex, while perception tasks are classified by high frequency EEG signals over occipital and frontal cortex. Combining data sets in training results in a balanced model improving classification of the imagination task without significantly changing performance in the visual task. Our approach not only improves performance and interpretability but also potentially reduces the burden on training since we can improve the accuracy of classifying a relatively hard task with high variability (imagination) by combining with the data of the relatively easy task, observing visual images.

EEG2IMAGE: Image Reconstruction from EEG Brain Signals

Neurocognition-inspired Design with Machine Learning

NeuroGAN: image reconstruction from EEG signals via an attention-based GAN

Visual Saliency and Image Reconstruction from EEG Signals via an Effective Geometric Deep Network-Based Generative Adversarial Network

Generating Visual Stimuli from EEG Recordings using Transformer-encoder based EEG encoder and GAN

Improving classification and reconstruction of imagined images from EEG signals

BrainDreamer: Reasoning-Coherent and Controllable Image Generation from EEG Brain Signals via Language Guidance

A GAN model encoded by CapsEEGNet for visual EEG encoding and image reproduction

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion

Generative Adversarial Network (GAN) for Simulating Electroencephalography

MindCeive: Perceiving human imagination using CNN-GRU and GANs

Human-in-the-loop design with machine learning

EEG to fMRI Synthesis: Is Deep Learning a candidate?

EEG-based Image Feature Extraction for Visual Classification using Deep Learning

A Visual Characteristic Land-Scape Design for EEG Signal Based on LSTM-GAN

Image classification and reconstruction from low-density EEG

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

Multi-Class Image Generation from EEG Features with Conditional Generative Adversarial Networks

Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models

Generative adversarial networks in EEG analysis: an overview

Reconstructing Visual Stimulus Images from EEG Signals Based on Deep Visual Representation Model