Abstract:Multivoxel pattern analysis (MVPA) examines the differences in multivoxel patterns evoked by different cognitive conditions using machine learning methods such as logistic regression and support vector machine. These methods are linear methods. It is possible that there are nonlinear relationships in the data that are not readily detected by these methods. We attempted to apply deep neural networks to address this problem. FMRI data were recorded from humans (n=20) performing a cued visual spatial/feature attention task in which an auditory cue instructed the subject to attend either left or right visual field (spatial trials), or attend either red or green color (feature trials). Following a random delay, two rectangular stimuli appeared, one in each visual field, and the subjects reported the orientation of the rectangle in the attended location (spatial trials) or with the attended color (feature trials). A deep neural network (DNN) was trained to take cue-evoked fMRI data as input features to predict trial labels. For feature (spatial) attention control, feature (spatial) trial data from 19 subjects were used to train a DNN model, which was then tested on the remaining subject. This process was repeated 20 times and the 20 decoding accuracies were averaged. We found that using the whole brain, the accuracy for decoding feature attention (cue red vs cue green) and spatial attention control (cue left vs cue right) is 59% and 61% respectively, both significantly above chance level of 50%. The heatmaps derived from the DNN models revealed important regions that contribute to both feature and spatial attention control as well as regions that contribute mainly to feature attention control or spatial attention control. In sum, DNNs can yield insights underlying attention control that complement other methods and provide a new approach for uncovering more complex relations between cognitive conditions and neural activities.

Decoding Disparity Categories in 3-Dimensional Images from Fmri Data Using Functional Connectivity Patterns

Disparity level identification using the voxel-wise Gabor model of fMRI data.

Stereoscopic processing of crossed and uncrossed disparities in the human visual cortex

Discrimination of Stereo Images by the Human Brain Based on fMRI

Multivoxel Pattern Selectivity for Perceptually Relevant Binocular Disparities in the Human Brain

Voxel-wise Binocular Energy Models for Binocular Disparity Decoding

Correlation and Matching Representations of Binocular Disparity across the Human Visual Cortex

Responses to interocular disparity correlation in the human cerebral cortex

Preliminary Fmri Research of the Human Brain Activation under Stereoscopic Vision

3D Contrast Image Reconstruction From Human Brain Activity

Functional specialization in human dorsal pathway for stereoscopic depth processing

Image Categorization from Functional Magnetic Resonance Imaging Using Functional Connectivity.

Functional diversity of visual cortex improves constraint-free natural image reconstruction from human brain activity

An ERP Research on the Cerebral Functional Regions Correlated to Stereoscopic Perception

Binocular Disparity: Neural Mechanisms and Perceptual Learning

Neural Correlates of Stereoscopic Depth Perception: A Fnirs Study

Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis

Encoding of 3 D surface priors in the spatial pattern of interaction among V 1 disparity neurons

A Novel Interpretable Fusion Analytic Framework for Investigating Functional Brain Connectivity Differences in Cognitive Impairments

Decoding Visual Feature Versus Visual Spatial Attention Control with Deep Neural Networks

Rapidly Decoding Image Categories from MEG Data Using a Multivariate Short-Time FC Pattern Analysis Approach