Abstract:Multi-modal Magnetic Resonance Imaging (MRI) is imperative for accurate brain tumor segmentation, offering indispensable complementary information. Nonetheless, the absence of modalities poses significant challenges in achieving precise segmentation. Recognizing the shared anatomical structures between mono-modal and multi-modal representations, it is noteworthy that mono-modal images typically exhibit limited features in specific regions and tissues. In response to this, we present Anatomical Consistency Distillation and Inconsistency Synthesis (ACDIS), a novel framework designed to transfer anatomical structures from multi-modal to mono-modal representations and synthesize modality-specific features. ACDIS consists of two main components: Anatomical Consistency Distillation (ACD) and Modality Feature Synthesis Block (MFSB). ACD incorporates the Anatomical Feature Enhancement Block (AFEB), meticulously mining anatomical information. Simultaneously, Anatomical Consistency ConsTraints (ACCT) are employed to facilitate the consistent knowledge transfer, i.e., the richness of information and the similarity in anatomical structure, ensuring precise alignment of structural features across mono-modality and multi-modality. Complementarily, MFSB produces modality-specific features to rectify anatomical inconsistencies, thereby compensating for missing information in the segmented features. Through validation on the BraTS2018 and BraTS2020 datasets, ACDIS substantiates its efficacy in the segmentation of brain tumors with missing MRI modalities.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the problem of how to achieve accurate brain tumor segmentation in the case of partial modality missing. Specifically, multi - modal Magnetic Resonance Imaging (Multi - modal MRI) is crucial for accurate brain tumor segmentation because it provides complementary information. However, in clinical practice, it is often not possible to ensure that all modalities are available simultaneously, which poses a challenge to accurate segmentation. For example, a patient may be allergic to contrast agents, resulting in the inability to obtain the T1c modality, or other modalities may be unavailable due to different MRI parameters. Therefore, how to perform brain tumor segmentation in the absence of certain modalities has become an important issue in clinical applications. ### Overview of the method To solve this problem, the authors proposed the **Anatomical Consistency Distillation and Inconsistency Synthesis (ACDIS)** framework. This framework consists of two main components: 1. **Anatomical Consistency Distillation (ACD)**: - **Anatomical Feature Enhancement Block (AFEB)**: Used to deeply extract the anatomical information of a single modality. - **Anatomical Consistency Constraints (ACCT)**: Ensure the anatomical structure consistency between multi - modality and single - modality through the calculation of variance and covariance in the local window. 2. **Modality Feature Synthesis Block (MFSB)**: - Used to generate the features of the missing modality and compensate for the anatomical structure inconsistency caused by different pixel intensities. ### Main contributions - **Studied the anatomical consistency and inconsistency between single - modality and multi - modality**, and based on this, proposed the ACDIS framework, which utilizes consistent anatomical insights and synthesizes inconsistent anatomical features. - **ACDIS combines ACD and MFSB**. Through the calculation of variance and covariance in the local window, it transmits rich information and emphasizes the similarity of anatomical structures, thereby enhancing the single - modality representation. MFSB is used to learn the modality - specific style and make up for the missing information. - **On the BraTS2018 and BraTS2020 datasets, ACDIS outperforms existing methods in the brain tumor segmentation task with missing modalities**. ### Experimental results - **Performance comparison**: Under different modality - missing combinations, the average Dice coefficient of ACDIS is significantly better than other state - of - the - art methods, increasing by 1.3%, 1.1% and 1.5% on the whole tumor (WT), tumor core (TC) and enhanced tumor (ET), respectively. - **Ablation experiment**: By removing the AFEB and MFSB modules, the effectiveness of each component was verified. The experimental results show that the combination of ACD and MFSB significantly improves the performance of the model. ### Conclusion This paper effectively solves the challenge of brain tumor segmentation in the case of partial modality missing by proposing the ACDIS framework, providing a new solution for clinical applications.

Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities

Mmformer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation

M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities

MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment

Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion

A Multimodal Feature Distillation with CNN-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities

Enhancing Incomplete Multi-modal Brain Tumor Segmentation with Intra-modal Asymmetry and Inter-modal Dependency

Adapting Segment Anything Model for 3D Brain Tumor Segmentation With Missing Modalities

A 3D Cross-Modality Feature Interaction Network with Volumetric Feature Alignment for Brain Tumor and Tissue Segmentation

FIMD: Fusion-Inspired Modality Distillation for Enhanced MRI Segmentation in Incomplete Multi-Modal Scenarios

Cross-modality Synthesis Aiding Lung Tumor Segmentation on Multi-Modal MRI Images

Multimodality-Assisted Semi-Supervised Brain Tumor Segmentation in Nondominant Modality Based on Consistency Learning

ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities

Adaptive Cross-Feature Fusion Network with Inconsistency Guidance for Multi-Modal Brain Tumor Segmentation.

TMFormer: Token Merging Transformer for Brain Tumor Segmentation with Missing Modalities

Multi‐modal Brain Tumor Image Segmentation Based on SDAE

Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis

Brain Tumor Segmentation on MRI with Missing Modalities

Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion

Deep Cascaded Attention Network for Multi-task Brain Tumor Segmentation.

Unpaired Multi-Modal Tumor Segmentation with Structure Adaptation