Abstract:Objective. Recently, deep learning models have been successfully applied in functional magnetic resonance imaging (fMRI) modeling and associated applications. However, there still exist at least two challenges. Firstly, due to the lack of sufficient data, deep learning models tend to suffer from overfitting in the training process. Secondly, it is still challenging to model the temporal dynamics from fMRI, due to that the brain state is continuously changing over scan time. In addition, existing methods rarely studied and applied fMRI data augmentation.Approach. In this work, we construct a deep recurrent variational auto-encoder (DRVAE) that combined variational auto-encoder and recurrent neural network, aiming to address all of the above mentioned challenges. The encoder of DRVAE can extract more generalized temporal features from assumed Gaussian distribution of input data, and the decoder of DRVAE can generate new data to increase training samples and thus partially relieve the overfitting issue. The recurrent layers in DRVAE are designed to effectively model the temporal dynamics of functional brain activities. LASSO (least absolute shrinkage and selection operator) regression is applied on the temporal features and input fMRI data to estimate the corresponding spatial networks.Main results. Extensive experimental results on seven tasks from HCP dataset showed that the DRVAE and LASSO framework can learn meaningful temporal patterns and spatial networks from both real data and generated data. The results on group-wise data and single subject suggest that the brain activities may follow certain distribution. Moreover, we applied DRVAE on four resting state fMRI datasets from ADHD-200 for data augmentation, and the results showed that the classification performances on augmented datasets have been considerably improved.Significance. The proposed method can not only derive meaningful temporal features and spatial networks from fMRI, but also generate high-quality new data for fMRI data augmentation and associated applications.

Explore The Hierarchical Auditory Information Processing Via Deep Convolutional Autoencoder

Exploring Hierarchical Auditory Representation via a Neural Encoding Model

Modeling Task Fmri Data Via Deep Convolutional Autoencoder

A hierarchical sparse coding model predicts acoustic feature encoding in both auditory midbrain and cortex

Dissecting neural computations in the human auditory pathway using deep neural networks for speech

Dissecting neural computations of the human auditory pathway using deep neural networks for speech

A Hybrid Autoencoder Framework of Dimensionality Reduction for Brain-Computer Interface Decoding

Investigating Self-Supervised Deep Representations for EEG-based Auditory Attention Decoding

Reverse the auditory processing pathway: Coarse-to-fine audio reconstruction from fMRI

The Unsupervised Hierarchical Convolutional Sparse Auto-Encoder for Neuroimaging Data Classification

Identify Hierarchical Structures from Task-Based fMRI Data via Hybrid Spatiotemporal Neural Architecture Search Net

Hierarchical Deep Feature Learning For Decoding Imagined Speech From EEG

Exploring the Functional Difference of Gyri/Sulci via Hierarchical Interpretable Autoencoder

Exploring Brain Hemodynamic Response Patterns Via Deep Recurrent Autoencoder

Inferring Mechanisms of Auditory Attentional Modulation with Deep Neural Networks

Constructing the hierarchy of predictive auditory sequences in the marmoset brain

Neural encoding and interpretation for high-level visual cortices based on fMRI using image caption features

Deep Variational Autoencoder for Mapping Functional Brain Networks

Modeling and augmenting of fMRI data using deep recurrent variational auto-encoder

Neural Encoding for Human Visual Cortex With Deep Neural Networks Learning “What” and “Where”

Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)