Ambiguous Medical Image Segmentation using Diffusion Models

Aimon Rahman,Jeya Maria Jose Valanarasu,Ilker Hacihaliloglu,Vishal M Patel

2023-04-11

Abstract:Collective insights from a group of experts have always proven to outperform an individual's best diagnostic for clinical tasks. For the task of medical image segmentation, existing research on AI-based alternatives focuses more on developing models that can imitate the best individual rather than harnessing the power of expert groups. In this paper, we introduce a single diffusion model-based approach that produces multiple plausible outputs by learning a distribution over group insights. Our proposed model generates a distribution of segmentation masks by leveraging the inherent stochastic sampling process of diffusion using only minimal additional learning. We demonstrate on three different medical image modalities- CT, ultrasound, and MRI that our model is capable of producing several possible variants while capturing the frequencies of their occurrences. Comprehensive results show that our proposed approach outperforms existing state-of-the-art ambiguous segmentation networks in terms of accuracy while preserving naturally occurring variation. We also propose a new metric to evaluate the diversity as well as the accuracy of segmentation predictions that aligns with the interest of clinical practice of collective insights.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper primarily addresses the issue of ambiguity in medical image segmentation. Specifically:

1. **Fusion of Multiple Expert Opinions**:
- Most current medical image segmentation models are deterministic, meaning each input image produces only one segmentation mask. However, in clinical practice, different diagnostic experts may have different opinions on the same image, leading to diversity in diagnostic results. To improve diagnostic accuracy, it is often necessary to integrate the opinions of multiple experts.

2. **Improvement of Fuzzy Segmentation Networks**:
- Existing fuzzy segmentation networks can generate multiple segmentation results, but the diversity and quality of these results are often unsatisfactory. For example, methods based on conditional variational autoencoders (c-VAE) inject randomness at the highest resolution, resulting in segmentation outcomes that are blurry and lack diversity.

3. **Application of Diffusion Models**:
- This paper proposes a new framework based on diffusion models—CIMD (Collectively Intelligent Medical Diffusion), which can generate multiple reasonable segmentation masks through a random sampling process without adding extra networks. This method not only improves the diversity of segmentation results but also maintains natural variations between segmentation masks.

4. **Proposal of New Evaluation Metrics**:
- Given that existing evaluation metrics (such as GED) cannot fully assess the performance of fuzzy segmentation models, a new evaluation metric, the CI Score (Collective Insight Score), is proposed. This metric combines factors such as sensitivity, consensus, and diversity, better meeting the needs of clinical practice.