Abstract:A crucial point in neuroscience is how to correctly decode cognitive information from brain dynamics for motion control and neural rehabilitation. However, due to the instability and high dimensions of electroencephalogram (EEG) recordings, it is difficult to directly obtain information from original data. Thus, in this work, we design visual experiments and propose a novel decoding method based on the neural manifold of cortical activity to find critical visual information. First, we studied four major frequency bands divided from EEG and found that the responses of the EEG alpha band (8–15 Hz) in the frontal and occipital lobes to visual stimuli occupy a prominent place. Besides, the essential features of EEG data in the alpha band are further mined via two manifold learning methods. We connect temporally consecutive brain states in the t distribution random adjacency embedded (t-SNE) map on the trial-by-trial level and find the brain state dynamics to form a cyclic manifold, with the different tasks forming distinct loops. Meanwhile, it is proved that the latent factors of brain activities estimated by t-SNE can be used for more accurate decoding and the stable neural manifold is found. Taking the latent factors of the manifold as independent inputs, a fuzzy system-based Takagi-Sugeno-Kang model is established and further trained to identify visual EEG signals. The combination of t-SNE and fuzzy learning can highly improve the accuracy of visual cognitive decoding to 81.98%. Moreover, by optimizing the features, it is found that the combination of the frontal lobe, the parietal lobe, and the occipital lobe is the most effective factor for visual decoding with 83.05% accuracy. This work provides a potential tool for decoding visual EEG signals with the help of low-dimensional manifold dynamics, especially contributing to the brain–computer interface (BCI) control, brain function research, and neural rehabilitation.

Decoding human brain activity with deep learning

Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion models

Deep Natural Image Reconstruction from Human Brain Activity Based on Conditional Progressively Growing Generative Adversarial Networks

Brain Decoding Using Deep Convolutional Network and Its Application in Cross-Subject Analysis.

Transfer Learning-Based Behavioural Task Decoding from Brain Activity

Neurodm: Decoding and Visualizing Human Brain Activity with Eeg-Guided Diffusion Model

A Visual Characteristic Land-Scape Design for EEG Signal Based on LSTM-GAN

Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision

fMRI Brain Decoding and Its Applications in Brain–Computer Interface: A Survey

Decoding Digital Visual Stimulation From Neural Manifold With Fuzzy Leaning on Cortical Oscillatory Dynamics

Gated Transformer for Decoding Human Brain EEG Signals

Decoding Natural Images from EEG for Object Recognition

Deep Learning Decoding of Mental State in Non-invasive Brain Computer Interface

Visual Image Decoding of Brain Activities Using a Dual Attention Hierarchical Latent Generative Network with Multiscale Feature Fusion

The brain-inspired decoder for natural visual image reconstruction

A Linear-Attention-combined Convolutional Neural Network for EEG-based Visual Stimulus Recognition

Brain decoding: toward real-time reconstruction of visual perception

Deep Learning Human Mind for Automated Visual Classification

Long short‐term memory‐based neural decoding of object categories evoked by natural images

Visual Image Decoding of Brain Activities using a Dual Attention Hierarchical Latent Generative Network with Multi-Scale Feature Fusion

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion