Abstract:<h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Background</h3><p>Deep neural networks have revolutionised machine learning, with unparalleled performance in object classification. However, in brain imaging (<em>e.g.</em>, fMRI), the direct application of Convolutional Neural Networks (CNN) to decoding subject states or perception from imaging data seems impractical given the scarcity of available data.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">New method</h3><p>In this work we propose a robust method to transfer information from deep learning (DL) features to brain fMRI data with the goal of decoding. By adopting Reduced Rank Regression with Ridge Regularisation we establish a multivariate link between imaging data and the fully connected layer (<span class="monospace">fc7</span>) of a CNN. We exploit the reconstructed <span class="monospace">fc7</span> features by performing an object image classification task on two datasets: one of the largest fMRI databases, taken from different scanners from more than two hundred subjects watching different movie clips, and another with fMRI data taken while watching static images.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Results</h3><p>The <span class="monospace">fc7</span> features could be significantly reconstructed from the imaging data, and led to significant decoding performance.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Comparison with existing methods</h3><p>The decoding based on reconstructed <span class="monospace">fc7</span> outperformed the decoding based on imaging data alone.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Conclusion</h3><p>In this work we show how to improve fMRI-based decoding benefiting from the mapping between functional data and CNN features. The potential advantage of the proposed method is twofold: the extraction of stimuli representations by means of an automatic procedure (unsupervised) and the embedding of high-dimensional neuroimaging data onto a space designed for visual object discrimination, leading to a more manageable space from dimensionality point of view.</p>

Transfer learning of deep neural network representations for fMRI decoding

Deep learning of fMRI big data: a novel approach to subject-transfer decoding

Brain Decoding Using fMRI Images for Multiple Subjects through Deep Learning

A Hybrid Spatio-Temporal Deep Belief Network and Sparse Representation-Based Framework Reveals Multi-Level Core Functional Components in Decoding Multi-Task fMRI Signals

Dimensionality Reduction of Local Field Potential Features with Convolution Neural Network in Neural Decoding: A Pilot Study

Evaluating deep transfer learning for whole-brain cognitive decoding

Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning

A Hybrid Autoencoder Framework of Dimensionality Reduction for Brain-Computer Interface Decoding

See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI

Decoding Visual fMRI Stimuli from Human Brain Based on Graph Convolutional Neural Network

Decoding and mapping task states of the human brain via deep learning

Transferability of Brain decoding using Graph Convolutional Networks

Deep learning approaches for neural decoding: from CNNs to LSTMs and spikes to fMRI

Attention module improves both performance and interpretability of four‐dimensional functional magnetic resonance imaging decoding neural network

Decoding Brain States From fMRI Signals by Using Unsupervised Domain Adaptation

Evaluating the structure of cognitive tasks with transfer learning

Cortical ROI Importance Improves MI Decoding From EEG Using Fused Light Neural Network

fMRI Brain Decoding and Its Applications in Brain–Computer Interface: A Survey

Evaluation of Task fMRI Decoding With Deep Learning on a Small Sample Dataset

Retrieving and reconstructing conceptually similar images from fMRI with latent diffusion models and a neuro-inspired brain decoding model

Group-level Brain Decoding with Deep Learning