Abstract:In this study, we proposed and validated a multi-atlas guided 3D fully convolutional network (FCN) ensemble model (M-FCN) for segmenting brain regions of interest (ROIs) from structural magnetic resonance images (MRIs). One major limitation of existing state-of-the-art 3D FCN segmentation models is that they often apply image patches of fixed size throughout training and testing, which may miss some complex tissue appearance patterns of different brain ROIs. To address this limitation, we trained a 3D FCN model for each ROI using patches of adaptive size and embedded outputs of the convolutional layers in the deconvolutional layers to further capture the local and global context patterns. In addition, with an introduction of multi-atlas based guidance in M-FCN, our segmentation was generated by combining the information of images and labels, which is highly robust. To reduce over-fitting of the FCN model on the training data, we adopted an ensemble strategy in the learning procedure. Evaluation was performed on two brain MRI datasets, aiming respectively at segmenting 14 subcortical and ventricular structures and 54 brain ROIs. The segmentation results of the proposed method were compared with those of a state-of-the-art multi-atlas based segmentation method and an existing 3D FCN segmentation model. Our results suggested that the proposed method had a superior segmentation performance.

What problem does this paper attempt to address?

The main problem this paper attempts to address is the limitations of existing 3D Fully Convolutional Networks (3D FCN) in brain region segmentation. Specifically, current 3D FCN models typically use fixed-size image patches throughout the training and testing process, which may lead to missing complex tissue appearance patterns in different brain regions. To overcome this limitation, the authors propose a multi-atlas guided 3D Fully Convolutional Network ensemble model (M-FCN) for segmenting regions of interest (ROIs) from structural Magnetic Resonance Imaging (MRI). ### The main contributions of the paper include: 1. **Adaptive Patch Size Determination Method**: An adaptive method is proposed to determine the patch size for different ROIs to better capture both local and global contextual patterns of different ROIs. 2. **Multi-Atlas Guidance**: By introducing multi-atlas guidance, combining image and label information, the generated segmentation results are more robust. 3. **Efficient 3D FCN Architecture**: The proposed 3D FCN is an end-to-end network consisting only of convolutional and deconvolutional layers, which can be efficiently trained in less than 10 epochs. ### Problems Addressed: - **Limitations of Fixed-Size Image Patches**: Existing methods use fixed-size image patches, which may fail to capture the complex tissue appearance patterns of different ROIs. - **Limitations of Single Atlas**: Single atlas methods may fail to capture inter-individual variability. - **Overfitting Problem**: The ensemble strategy reduces the overfitting of the FCN model on the training data. ### Experimental Validation: - **Datasets**: The paper validates the method on two brain MRI datasets. The first dataset contains 16 high-resolution T1-weighted 3D volumetric MPRAGE images for segmenting 14 subcortical and lateral ventricle structures. The second dataset contains 40 brain images for segmenting 54 brain regions (excluding the brainstem and cerebellum). - **Evaluation Method**: A 4-fold cross-validation strategy is adopted, comparing the segmentation results with existing multi-atlas segmentation methods and 3D FCN segmentation models. The results show that the proposed method has superior segmentation performance. In summary, this paper effectively addresses the limitations of existing 3D FCNs in brain region segmentation by proposing a multi-atlas guided 3D Fully Convolutional Network ensemble model, improving segmentation accuracy and robustness.

Brain segmentation based on multi-atlas guided 3D fully convolutional network ensembles

Brain Image Labeling Using Multi-atlas Guided 3D Fully Convolutional Networks

Brain Image Parcellation Using Multi-Atlas Guided Adversarial Fully Convolutional Network.

Multi-stream 3D FCN with multi-scale deep supervision for multi-modality isointense infant brain MR image segmentation

APRNet: A 3D Anisotropic Pyramidal Reversible Network with Multi-Modal Cross-Dimension Attention for Brain Tissue Segmentation in MR Images.

Combined 3D CNN for Brain Tumor Segmentation

Robust 3D Convolutional Neural Network with Boundary Correction for Accurate Brain Tissue Segmentation.

FD-FCN: 3D Fully Dense and Fully Convolutional Network for Semantic Segmentation of Brain Anatomy

Automatic Brain Labeling Via Multi-Atlas Guided Fully Convolutional Networks.

Integrated 3d flow-based multi-atlas brain structure segmentation.

Supervoxel Based Method for Multi-Atlas Segmentation of Brain MR Images

Brain Tumor Segmentation Using a Fully Convolutional Neural Network with Conditional Random Fields.

AFPNet: A 3D Fully Convolutional Neural Network with Atrous-Convolution Feature Pyramid for Brain Tumor Segmentation Via MRI Images.

Multi-Atlas Brain Parcellation Using Squeeze-and-Excitation Fully Convolutional Networks

A deep learning model integrating FCNNs and CRFs for brain tumor segmentation

Multi-scale Strategy Based 3D Dual-Encoder Brain Tumor Segmentation Network with Attention Mechanism.

Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation

IOUC-3DSFCNN: Segmentation of Brain Tumors via IOU Constraint 3D Symmetric Full Convolution Network with Multimodal Auto-context

Multi-Atlas Based Segmentation of Brainstem Nuclei from MR Images by Deep Hyper-Graph Learning

Learning U-Net Based Multi-Scale Features in Encoding-Decoding for MR Image Brain Tissue Segmentation.

3D EMSU‐Net: A Framework for Automatic Segmentation of Brain Tumors