Abstract:Objectives: To build self-supervised foundation models for multicontrast MRI of the whole brain and evaluate their efficacy in assisting diagnosis of brain tumors. Methods: In this retrospective study, foundation models were developed using 57,621 enhanced head MRI scans through self-supervised learning with a pretext task of cross-contrast context restoration with two different content dropout schemes. Downstream classifiers were constructed based on the pretrained foundation models and fine-tuned for brain tumor detection, discrimination, and molecular status prediction. Metrics including accuracy, sensitivity, specificity, and area under the ROC curve (AUC) were used to evaluate the performance. Convolutional neural networks trained exclusively on downstream task data were employed for comparative analysis. Results: The pretrained foundation models demonstrated their ability to extract effective representations from multicontrast whole-brain volumes. The best classifiers, endowed with pretrained weights, showed remarkable performance with accuracies of 94.9, 92.3, and 80.4%, and corresponding AUC values of 0.981, 0.972, and 0.852 on independent test datasets in brain tumor detection, discrimination, and molecular status prediction, respectively. The classifiers with pretrained weights outperformed the convolutional classifiers trained from scratch by approximately 10% in terms of accuracy and AUC across all tasks. The saliency regions in the correctly predicted cases are mainly clustered around the tumors. Classifiers derived from the two dropout schemes differed significantly only in the detection of brain tumors. Conclusions: Foundation models obtained from self-supervised learning have demonstrated encouraging potential for scalability and interpretability in downstream brain tumor-related tasks and hold promise for extension to neurological diseases with diffusely distributed lesions. Clinical relevance statement: The application of our proposed method to the prediction of key molecular status in gliomas is expected to improve treatment planning and patient outcomes. Additionally, the foundation model we developed could serve as a cornerstone for advancing AI applications in the diagnosis of brain-related diseases.

BrainFounder: Towards Brain Foundation Models for Neuroimage Analysis

BrainSegFounder: Towards 3D foundation models for neuroimage segmentation

A foundation model for generalized brain MRI analysis

Medical image foundation models in assisting diagnosis of brain tumors: a pilot study

BrainWave: A Brain Signal Foundation Model for Clinical Applications

BrainMass: Advancing Brain Network Analysis for Diagnosis with Large-scale Self-Supervised Learning

NeuroAtlas: An Artificial Intelligence-based Framework for Annotation, Segmentation and Registration of Large Scale Biomedical Imaging Data

BrainLM: A foundation model for brain activity recordings

UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training

Anatomical Foundation Models for Brain MRIs

Brain3D: Generating 3D Objects from fMRI

Brainchop: Providing an Edge Ecosystem for Deployment of Neuroimaging Artificial Intelligence Models

Brain-ID: Learning Contrast-agnostic Anatomical Representations for Brain Imaging

Utilizing Deep Learning to Improve Diagnostic Accuracy in Glioma, Pituitary Tumors, and Meningiomas

Automated brain extraction of multi-sequence MRI using artificial neural networks

Generative models of MRI-derived neuroimaging features and associated dataset of 18,000 samples

Foundation model of neural activity predicts response to new stimulus types and anatomy

GSFormer: A Gated Skip Connection and Feature Fusion Transformer-based Neural Network for 3D MRI Brain Tumor Segmentation

Automated Ensemble-Based Segmentation of Adult Brain Tumors: A Novel Approach Using the BraTS AFRICA Challenge Data

Neuro-GPT: Towards A Foundation Model for EEG