Abstract:Multi-modal brain magnetic resonance imaging (MRI) data has been widely applied in vison-based brain tumor segmentation methods due to its complementary diagnostic information from different modalities. Since the multi-modal image data is likely to be corrupted by noise or artifacts during the practical scanning process, making it difficult to build a universal model for the subsequent segmentation and diagnosis with incomplete input data, image completion has become one of the most attractive fields in the medical image pre-processing. It can not only assist clinicians to observe the patient's lesion area more intuitively and comprehensively, but also realize the desire to save costs for patients and reduce the psychological pressure of patients during tedious pathological examinations. Recently, many deep learning-based methods have been proposed to complement the multi-modal image data and provided good performance. However, current methods cannot fully reflect the continuous semantic information between the adjacent slices and the structural information of the intra-slice features, resulting in limited complementation effects and efficiencies. To solve these problems, in this work, we propose a novel generative adversarial network (GAN) framework, named as random generative adversarial network (RAGAN), to complete the missing T1, T1ce, and FLAIR data from the given T2 modal data in real brain MRI, which consists of the following parts: (1) For the generator, we use T2 modal images and multi-modal classification labels from the same sample for cyclically supervised training of image generation, so as to realize the restoration of arbitrary modal images. (2) For the discriminator, a multi-branch network is proposed where the primary branch is designed to judge whether the certain generated modal image is similar to the target modal image, while the auxiliary branch is to judge whether its essential visual features are similar to those of the target modal image. We conduct qualitative and quantitative experimental validations on the BraTs2018 dataset, generating 10,686 MRI data in each missing modality. Real brain tumor morphology images were compared with synthetic brain tumor morphology images using PSNR and SSIM as evaluation metrics. Experiments demonstrate that the brightness, resolution, location, and morphology of brain tissue under different modalities are well reconstructed. Meanwhile, we also use the segmentation network as a further validation experiment. Blend synthetic and real images into a segmentation network. Our segmentation network adopts the classic segmentation network UNet. The segmentation result is 77.58%. In order to prove the value of our proposed method, we use the better segmentation network RES_UNet with depth supervision as the segmentation model, and the segmentation accuracy rate is 88.76%. Although our method does not significantly outperform other algorithms, the DICE value is 2% higher than the current state-of-the-art data completion algorithm TC-MGAN.

Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans

Improvement of Multiparametric MR Image Segmentation by Augmenting the Data With Generative Adversarial Networks for Glioma Patients

MMGan: a Multimodal MR Brain Tumor Image Segmentation Method

Ensemble Learning and 3D Pix2Pix for Comprehensive Brain Tumor Analysis in Multimodal MRI

Medical image synthesis via conditional GANs: Application to segmenting brain tumours

Brain Tumor Segmentation with Generative Adversarial Nets

High Tissue Contrast MRI Synthesis Using Multi-Stage Attention-GAN for Glioma Segmentation

Using a generative adversarial network to generate synthetic MRI images for multi-class automatic segmentation of brain tumors

Multi-modal brain tumor segmentation via conditional synthesis with Fourier domain adaptation

Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion models

Multi-Modal Brain Tumor Data Completion Based on Reconstruction Consistency Loss

3D Deep Neural Network-Based Brain Tumor Segmentation Using Multimodality Magnetic Resonance Sequences.

Segmentation of Brain Gliomas Based on a Multi-modal Multi-scale Double-pathway 3D Residual CNN

Feature-enhanced generation and multi-modality fusion based deep neural network for brain tumor segmentation with missing MR modalities

Brain Tumor Segmentation from MRI Using Pre-segmentation Based on Superpixels and Fully Convolutional Neural Networks

A Transformer-based Generative Adversarial Network for Brain Tumor Segmentation

Brain Multi-Parametric MRI Tumor Subregion Segmentation Via Hierarchical Substructural Activation Network.

Brain tumor image pixel segmentation and detection using an aggregation of GAN models with vision transformer

Segmentation of Gliomas Based on a Double-Pathway Residual Convolution Neural Network Using Multi-Modality Information

Generating 3D Brain Tumor Regions in MRI using Vector-Quantization Generative Adversarial Networks

Nimg-65. Synthetic Mri Generation To Facilitate Autosegmentation Of Pediatric Brain Tumors In Limited Data Scenarios