Abstract:The burgeoning field of brain health research increasingly leverages artificial intelligence (AI) to analyze and interpret neuroimaging data. Medical foundation models have shown promise of superior performance with better sample efficiency. This work introduces a novel approach towards creating 3-dimensional (3D) medical foundation models for multimodal neuroimage segmentation through self-supervised training. Our approach involves a novel two-stage pretraining approach using vision transformers. The first stage encodes anatomical structures in generally healthy brains from the large-scale unlabeled neuroimage dataset of multimodal brain magnetic resonance imaging (MRI) images from 41,400 participants. This stage of pertaining focuses on identifying key features such as shapes and sizes of different brain structures. The second pretraining stage identifies disease-specific attributes, such as geometric shapes of tumors and lesions and spatial placements within the brain. This dual-phase methodology significantly reduces the extensive data requirements usually necessary for AI model training in neuroimage segmentation with the flexibility to adapt to various imaging modalities. We rigorously evaluate our model, BrainSegFounder, using the Brain Tumor Segmentation (BraTS) challenge and Anatomical Tracings of Lesions After Stroke v2.0 (ATLAS v2.0) datasets. BrainSegFounder demonstrates a significant performance gain, surpassing the achievements of the previous winning solutions using fully supervised learning. Our findings underscore the impact of scaling up both the model complexity and the volume of unlabeled training data derived from generally healthy brains. Both of these factors enhance the accuracy and predictive capabilities of the model in neuroimage segmentation tasks. Our pretrained models and code are at https://github.com/lab-smile/BrainSegFounder.

BrainSegFounder: Towards 3D foundation models for neuroimage segmentation

BrainFounder: Towards Brain Foundation Models for Neuroimage Analysis

3D Brainformer: 3D Fusion Transformer for Brain Tumor Segmentation

Optimizing Brain Tumor Segmentation with MedNeXt: BraTS 2024 SSA and Pediatrics

Brain Tumor Segmentation Using an Ensemble of 3D U-Nets and Overall Survival Prediction Using Radiomic Features

DeepBrainSeg: Automated Brain Region Segmentation for Micro-Optical Images with a Convolutional Neural Network

Automated Ensemble-Based Segmentation of Adult Brain Tumors: A Novel Approach Using the BraTS AFRICA Challenge Data

TransSea: Hybrid CNN-Transformer with Semantic Awareness for 3D Brain Tumor Segmentation

Brain MRI Segmentation using Template-Based Training and Visual Perception Augmentation

GSFormer: A Gated Skip Connection and Feature Fusion Transformer-based Neural Network for 3D MRI Brain Tumor Segmentation

A deep learning model integrating FCNNs and CRFs for brain tumor segmentation

TransSea: Hybrid CNN–Transformer With Semantic Awareness for 3-D Brain Tumor Segmentation

Fully Automated Segmentation of Brain Tumor from Multiparametric MRI Using 3D Context U-Net with Deep Supervision.

nnSegNeXt: A 3D Convolutional Network for Brain Tissue Segmentation Based on Quality Evaluation

3D dense connectivity network with atrous convolutional feature pyramid for brain tumor segmentation in magnetic resonance imaging of human heads

On Enhancing Brain Tumor Segmentation Across Diverse Populations with Convolutional Neural Networks

A 3D Spatially Weighted Network for Segmentation of Brain Tissue from MRI

Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI

A novel method for glioma segmentation and classification on pre-operative MRI scans using 3D U-Nets and transfer learning

Attention-Enhanced Hybrid Feature Aggregation Network for 3D Brain Tumor Segmentation

A Deep-Learning Model with Learnable Group Convolution and Deep Supervision for Brain Tumor Segmentation