Abstract:In the diverse field of medical imaging, automatic segmentation has numerous applications and must handle a wide variety of input domains, such as different types of Computed Tomography (CT) scans and Magnetic Resonance (MR) images. This heterogeneity challenges automatic segmentation algorithms to maintain consistent performance across different modalities due to the requirement for spatially aligned and paired images. Typically, segmentation models are trained using a single modality, which limits their ability to generalize to other types of input data without employing transfer learning techniques. Additionally, leveraging complementary information from different modalities to enhance segmentation precision often necessitates substantial modifications to popular encoder-decoder designs, such as introducing multiple branched encoding or decoding paths for each modality. In this work, we propose a simple Multi-Modal Segmentation (MulModSeg) strategy to enhance medical image segmentation across multiple modalities, specifically CT and MR. It incorporates two key designs: a modality-conditioned text embedding framework via a frozen text encoder that adds modality awareness to existing segmentation frameworks without significant structural modifications or computational overhead, and an alternating training procedure that facilitates the integration of essential features from unpaired CT and MR inputs. Through extensive experiments with both Fully Convolutional Network and Transformer-based backbones, MulModSeg consistently outperforms previous methods in segmenting abdominal multi-organ and cardiac substructures for both CT and MR modalities. The code is available in this {\href{<a class="link-external link-https" href="https://github.com/ChengyinLee/MulModSeg_2024" rel="external noopener nofollow">this https URL</a>}{link}}.

Multi-panel Medical Image Segmentation Framework for Image Retrieval System

Multistage and Multi-features Medical Image Retrieval System

Correction To: A Simple and Effective Sub-Image Separation Method

An interactive medical image segmentation framework using iterative refinement

Partition-A-Medical-Image: Extracting Multiple Representative Subregions for Few-Shot Medical Image Segmentation

Partition-A-Medical-Image: Extracting Multiple Representative Sub-regions for Few-shot Medical Image Segmentation

Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine

An Efficient Two-Objective Automatic Sar Image Segmentation Framework Using Artificial Immune System

PIS-Net: Efficient Medical Image Segmentation Network with Multivariate Downsampling for Point-of-Care

MedSeq: Semantic Segmentation for Medical Image Sequences

A Mutual Inclusion Mechanism for Precise Boundary Segmentation in Medical Images

An Efficient Interactive Multilayered Approach for Segmentation of Complex Images

Multi-Modal Evaluation Approach for Medical Image Segmentation

Few-Shot Medical Image Segmentation via Generating Multiple Representative Descriptors

Multi-threshold image segmentation using a multi-strategy shuffled frog leaping algorithm

Fuzzy Clustering with Multi-Resolution Bilateral Filtering for Medical Image Segmentation

Image Segmentation Framework Based on Multiple Feature Spaces

MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training

A Dimension Hybrid Framework for Multimodal Medical Image Segmentation

Exploring Weakly Supervised Semantic Segmentation Ensembles for Medical Imaging Systems

Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes