Abstract:Multivoxel pattern analysis (MVPA) examines the differences in multivoxel patterns evoked by different cognitive conditions using machine learning methods such as logistic regression and support vector machine. These methods are linear methods. It is possible that there are nonlinear relationships in the data that are not readily detected by these methods. We attempted to apply deep neural networks to address this problem. FMRI data were recorded from humans (n=20) performing a cued visual spatial/feature attention task in which an auditory cue instructed the subject to attend either left or right visual field (spatial trials), or attend either red or green color (feature trials). Following a random delay, two rectangular stimuli appeared, one in each visual field, and the subjects reported the orientation of the rectangle in the attended location (spatial trials) or with the attended color (feature trials). A deep neural network (DNN) was trained to take cue-evoked fMRI data as input features to predict trial labels. For feature (spatial) attention control, feature (spatial) trial data from 19 subjects were used to train a DNN model, which was then tested on the remaining subject. This process was repeated 20 times and the 20 decoding accuracies were averaged. We found that using the whole brain, the accuracy for decoding feature attention (cue red vs cue green) and spatial attention control (cue left vs cue right) is 59% and 61% respectively, both significantly above chance level of 50%. The heatmaps derived from the DNN models revealed important regions that contribute to both feature and spatial attention control as well as regions that contribute mainly to feature attention control or spatial attention control. In sum, DNNs can yield insights underlying attention control that complement other methods and provide a new approach for uncovering more complex relations between cognitive conditions and neural activities.

Design of a High-Speed Superconducting Bearingless Machine for Flywheel Energy Storage Systems

Long short‐term memory‐based neural decoding of object categories evoked by natural images

‘When’ and ‘what’ did you see? A novel fMRI-based visual decoding framework

Efficient Neural Decoding Based on Multimodal Training

Category Decoding of Visual Stimuli From Human Brain Activity Using a Bidirectional Recurrent Neural Network to Simulate Bidirectional Information Flows in Human Visual Cortices

Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition

Decoding Visual Feature Versus Visual Spatial Attention Control with Deep Neural Networks

Generic decoding of seen and imagined objects using hierarchical visual features

Convolutional neural network-based encoding and decoding of visual object recognition in space and time

Brain-aligning of semantic vectors improves neural decoding of visual stimuli

Hierarchical Neural Representation of Dreamed Objects Revealed by Brain Decoding with Deep Neural Network Features

Investigation of turbomixers in continuous flow analysis.

Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision

Multi-Semantic Decoding of Visual Perception with Graph Neural Networks

Decoding human brain activity with deep learning

Decoding Visual Experience and Mapping Semantics through Whole-Brain Analysis Using fMRI Foundation Models

Temporal Variabilities Provide Additional Category-Related Information in Object Category Decoding: A Systematic Comparison of Informative EEG Features

Brain decoding: toward real-time reconstruction of visual perception

Deep learning of fMRI big data: a novel approach to subject-transfer decoding

Decoding Brain Signals from Rapid-Event EEG for Visual Analysis Using Deep Learning

From Sight to Insight: A Multi-task Approach with the Visual Language Decoding Model