Abstract:A crucial point in neuroscience is how to correctly decode cognitive information from brain dynamics for motion control and neural rehabilitation. However, due to the instability and high dimensions of electroencephalogram (EEG) recordings, it is difficult to directly obtain information from original data. Thus, in this work, we design visual experiments and propose a novel decoding method based on the neural manifold of cortical activity to find critical visual information. First, we studied four major frequency bands divided from EEG and found that the responses of the EEG alpha band (8–15 Hz) in the frontal and occipital lobes to visual stimuli occupy a prominent place. Besides, the essential features of EEG data in the alpha band are further mined via two manifold learning methods. We connect temporally consecutive brain states in the t distribution random adjacency embedded (t-SNE) map on the trial-by-trial level and find the brain state dynamics to form a cyclic manifold, with the different tasks forming distinct loops. Meanwhile, it is proved that the latent factors of brain activities estimated by t-SNE can be used for more accurate decoding and the stable neural manifold is found. Taking the latent factors of the manifold as independent inputs, a fuzzy system-based Takagi-Sugeno-Kang model is established and further trained to identify visual EEG signals. The combination of t-SNE and fuzzy learning can highly improve the accuracy of visual cognitive decoding to 81.98%. Moreover, by optimizing the features, it is found that the combination of the frontal lobe, the parietal lobe, and the occipital lobe is the most effective factor for visual decoding with 83.05% accuracy. This work provides a potential tool for decoding visual EEG signals with the help of low-dimensional manifold dynamics, especially contributing to the brain–computer interface (BCI) control, brain function research, and neural rehabilitation.

Decoding Action Observation Using Complex Brain Networks From Simultaneously Recorded Eeg-Fnirs Signals

Neural Activity and Decoding of Action Observation Using Combined EEG and fNIRS Measurement

Decoding the Nonstationary Neural Activity in Motor Cortex for Brain Machine Interfaces

A Study on Combining Local Field Potential and Single Unit Activity for Better Neural Decoding

Effective Connectivity Analysis and Classification of Action Observation From Different Perspectives: An fMRI Study

Feature Extraction and Classification Algorithm of Brain-computer Interface Based on Human Brain Central Nervous System

Temporal-Spatial Features of Intention Understanding Based on EEG-fNIRS Bimodal Measurement

Decoding Human Interaction Type from Inter-brain Synchronization by Using EEG Brain Network.

Motor Intention Decoding from the Upper Limb by Graph Convolutional Network Based on Functional Connectivity

Decoding human brain activity with deep learning

Brain Decoding Using fNIRS

Real-time recognition of different imagined actions on the same side of a single limb based on the fNIRS correlation coefficient

Classification of Intention Understanding Using EEG-NIRS Bimodal System

A Hybrid BCI Approach to Detect Brain Switch in Action Observation by Utilizing Convolution Neural Network.

Classification Algorithm for fNIRS-based Brain Signals Using Convolutional Neural Network with Spatiotemporal Feature Extraction Mechanism

Decoding Digital Visual Stimulation From Neural Manifold With Fuzzy Leaning on Cortical Oscillatory Dynamics

Unsupervised decoding of long-term, naturalistic human neural recordings with automated video and audio annotations

Motor Imagery Decoding Enhancement Based on Hybrid EEG-fNIRS Signals

Simultaneous multimodal fNIRS-EEG recordings reveal new insights in neural activity during motor execution, observation, and imagery

A Hybrid EEG-fNIRS Brain-Computer Interface Based on Dynamic Functional Connectivity and Long Short-Term Memory

fMRI Brain Decoding and Its Applications in Brain–Computer Interface: A Survey