Diversified and Personalized Multi-rater Medical Image Segmentation

Yicheng Wu,Xiangde Luo,Zhe Xu,Xiaoqing Guo,Lie Ju,Zongyuan Ge,Wenjun Liao,Jianfei Cai

2024-03-20

Abstract:Annotation ambiguity due to inherent data uncertainties such as blurred boundaries in medical scans and different observer expertise and preferences has become a major obstacle for training deep-learning based medical image segmentation models. To address it, the common practice is to gather multiple annotations from different experts, leading to the setting of multi-rater medical image segmentation. Existing works aim to either merge different annotations into the "groundtruth" that is often unattainable in numerous medical contexts, or generate diverse results, or produce personalized results corresponding to individual expert raters. Here, we bring up a more ambitious goal for multi-rater medical image segmentation, i.e., obtaining both diversified and personalized results. Specifically, we propose a two-stage framework named D-Persona (first Diversification and then Personalization). In Stage I, we exploit multiple given annotations to train a Probabilistic U-Net model, with a bound-constrained loss to improve the prediction diversity. In this way, a common latent space is constructed in Stage I, where different latent codes denote diversified expert opinions. Then, in Stage II, we design multiple attention-based projection heads to adaptively query the corresponding expert prompts from the shared latent space, and then perform the personalized medical image segmentation. We evaluated the proposed model on our in-house Nasopharyngeal Carcinoma dataset and the public lung nodule dataset (i.e., LIDC-IDRI). Extensive experiments demonstrated our D-Persona can provide diversified and personalized results at the same time, achieving new SOTA performance for multi-rater medical image segmentation. Our code will be released at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper addresses the issue of annotation ambiguity in multi-annotator medical image segmentation. Training deep learning models for medical image segmentation faces challenges due to data uncertainty (such as fuzzy boundaries) and variations in observer expertise and personal preferences. To solve this problem, the paper proposes a two-stage framework called D-Persona. Firstly, a probabilistic U-Net model is used to increase the diversity in predictions. Then, an attention projection head is designed to adaptively extract expert cues in the shared latent space for personalized medical image segmentation. Experimental results demonstrate that D-Persona can provide both diverse and personalized segmentation results, reaching a new level in multi-annotator medical image segmentation.

Diversified and Personalized Multi-rater Medical Image Segmentation

Multi-rater Prism: Learning self-calibrated medical image segmentation from multiple raters

Many Birds, One Stone: Medical Image Segmentation with Multiple Partially Labeled Datasets

Multi-rater Prompting for Ambiguous Medical Image Segmentation

Learning self-calibrated optic disc and cup segmentation from multi-rater annotations

Customized T-time inner sampling network with uncertainty-aware data augmentation strategy for multi-annotated lesion segmentation

Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation

Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks

Few-Shot Medical Image Segmentation via Generating Multiple Representative Descriptors

Calibrate the Inter-Observer Segmentation Uncertainty Via Diagnosis-First Principle.

MedUHIP: Towards Human-In-the-Loop Medical Segmentation

Segmentation with Multiple Acceptable Annotations: A Case Study of Myocardial Segmentation in Contrast Echocardiography

Learning Robust Medical Image Segmentation from Multi-source Annotations

Model-Heterogeneous Semi-Supervised Federated Learning for Medical Image Segmentation

MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling

MSE-Nets: Multi-annotated Semi-supervised Ensemble Networks for Improving Segmentation of Medical Image with Ambiguous Boundaries

A Multi-Objective Segmentation Method for Chest X-rays Based on Collaborative Learning from Multiple Partially Annotated Datasets

DPP-Net: Difficulty Perception-Processing Heterogeneous Network for Semi-supervised Medical Image Segmentation

Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction

Segment Together: A Versatile Paradigm for Semi-Supervised Medical Image Segmentation