Abstract:Introduction: Decoding brain activities is one of the most popular topics in neuroscience in recent years. And deep learning has shown high performance in fMRI data classification and regression, but its requirement for large amounts of data conflicts with the high cost of acquiring fMRI data. Methods: In this study, we propose an end-to-end temporal contrastive self-supervised learning algorithm, which learns internal spatiotemporal patterns within fMRI and allows the model to transfer learning to datasets of small size. For a given fMRI signal, we segmented it into three sections: the beginning, middle, and end. We then utilized contrastive learning by taking the end-middle (i.e., neighboring) pair as the positive pair, and the beginning-end (i.e., distant) pair as the negative pair. Results: We pretrained the model on 5 out of 7 tasks from the Human Connectome Project (HCP) and applied it in a downstream classification of the remaining two tasks. The pretrained model converged on data from 12 subjects, while a randomly initialized model required 100 subjects. We then transferred the pretrained model to a dataset containing unpreprocessed whole-brain fMRI from 30 participants, achieving an accuracy of 80.2 ± 4.7%, while the randomly initialized model failed to converge. We further validated the model's performance on the Multiple Domain Task Dataset (MDTB), which contains fMRI data of 26 tasks from 24 participants. Thirteen tasks of fMRI were selected as inputs, and the results showed that the pre-trained model succeeded in classifying 11 of the 13 tasks. When using the 7 brain networks as input, variations of the performance were observed, with the visual network performed as well as whole brain inputs, while the limbic network almost failed in all 13 tasks. Discussion: Our results demonstrated the potential of self-supervised learning for fMRI analysis with small datasets and unpreprocessed data, and for analysis of the correlation between regional fMRI activity and cognitive tasks.

Learning to segment with limited annotations: Self-supervised pretraining with regression and contrastive loss in MRI

Contrastive learning of global and local features for medical image segmentation with limited annotations

Self-Supervised Pretraining for 2D Medical Image Segmentation

Contrastive Pretraining for Echocardiography Segmentation with Limited Data

Semi-Supervised and Self-Supervised Collaborative Learning for Prostate 3D MR Image Segmentation

Leveraging Unlabeled Data for 3D Medical Image Segmentation through Self-Supervised Contrastive Learning

Positional Information is a Strong Supervision for Volumetric Medical Image Segmentation

A General Global and Local Pre-Training Framework for 3D Medical Image Segmentation.

CL-MRI: Self-Supervised Contrastive Learning to Improve the Accuracy of Undersampled MRI Reconstruction

Localized Region Contrast for Enhancing Self-Supervised Learning in Medical Image Segmentation

SELF-SUPERVISED LEARNING WITH RADIOLOGY REPORTS, A COMPARATIVE ANALYSIS OF STRATEGIES FOR LARGE VESSEL OCCLUSION AND BRAIN CTA IMAGES

Self-Ensembling Contrastive Learning for Semi-Supervised Medical Image Segmentation

A survey of the impact of self-supervised pretraining for diagnostic tasks in medical X-ray, CT, MRI, and ultrasound

Self-supervised pretraining improves the performance of classification of task functional magnetic resonance imaging

Improving Data-Efficiency and Robustness of Medical Imaging Segmentation Using Inpainting-Based Self-Supervised Learning

Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation

Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina

A review of self‐supervised, generative, and few‐shot deep learning methods for data‐limited magnetic resonance imaging segmentation

A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images

Collaborative Learning for Annotation-Efficient Volumetric MR Image Segmentation

Multi-level Asymmetric Contrastive Learning for Medical Image Segmentation Pre-training