A Generative Self-Supervised Framework using Functional Connectivity in fMRI Data

Jungwon Choi,Seongho Keum,EungGu Yun,Byung-Hoon Kim,Juho Lee

2023-12-05

Abstract:Deep neural networks trained on Functional Connectivity (FC) networks extracted from functional Magnetic Resonance Imaging (fMRI) data have gained popularity due to the increasing availability of data and advances in model architectures, including Graph Neural Network (GNN). Recent research on the application of GNN to FC suggests that exploiting the time-varying properties of the FC could significantly improve the accuracy and interpretability of the model prediction. However, the high cost of acquiring high-quality fMRI data and corresponding phenotypic labels poses a hurdle to their application in real-world settings, such that a model naïvely trained in a supervised fashion can suffer from insufficient performance or a lack of generalization on a small number of data. In addition, most Self-Supervised Learning (SSL) approaches for GNNs to date adopt a contrastive strategy, which tends to lose appropriate semantic information when the graph structure is perturbed or does not leverage both spatial and temporal information simultaneously. In light of these challenges, we propose a generative SSL approach that is tailored to effectively harness spatio-temporal information within dynamic FC. Our empirical results, experimented with large-scale (>50,000) fMRI datasets, demonstrate that our approach learns valuable representations and enables the construction of accurate and robust models when fine-tuned for downstream tasks.

Machine Learning,Computer Vision and Pattern Recognition,Image and Video Processing,Neurons and Cognition

What problem does this paper attempt to address?

The paper attempts to address the challenges faced when using functional connectivity (FC) for deep neural network modeling in functional magnetic resonance imaging (fMRI) data. Specifically:

1. **High data acquisition cost**: Acquiring high-quality fMRI data and their corresponding phenotypic labels is expensive, leading to insufficient performance or poor generalization ability of supervised learning models in practical applications.
2. **Limitations of self-supervised learning methods**: Most existing graph self-supervised learning (SSL) methods adopt contrastive strategies, which are prone to losing semantic information when the graph structure is perturbed and fail to utilize both spatial and temporal information simultaneously.

To address these issues, the authors propose a generative self-supervised learning framework—Spatio-Temporal Masked Autoencoder (ST-MAE), specifically designed to handle dynamic functional connectivity graphs. This method effectively captures the spatio-temporal information in dynamic FC, thereby constructing accurate and robust models for downstream tasks. Experimental results show that this method performs excellently on large-scale fMRI datasets and is particularly suitable for scenarios with limited labeled data, such as tasks involving the classification of mental disorders.