vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation

Bastian Wittmann,Yannick Wattenberg,Tamaz Amiranashvili,Suprosanna Shit,Bjoern Menze
2024-11-26
Abstract:Segmenting 3D blood vessels is a critical yet challenging task in medical image analysis. This is due to significant imaging modality-specific variations in artifacts, vascular patterns and scales, signal-to-noise ratios, and background tissues. These variations, along with domain gaps arising from varying imaging protocols, limit the generalization of existing supervised learning-based methods, requiring tedious voxel-level annotations for each dataset separately. While foundation models promise to alleviate this limitation, they typically fail to generalize to the task of blood vessel segmentation, posing a unique, complex problem. In this work, we present vesselFM, a foundation model designed specifically for the broad task of 3D blood vessel segmentation. Unlike previous models, vesselFM can effortlessly generalize to unseen domains. To achieve zero-shot generalization, we train vesselFM on three heterogeneous data sources: a large, curated annotated dataset, data generated by a domain randomization scheme, and data sampled from a flow matching-based generative model. Extensive evaluations show that vesselFM outperforms state-of-the-art medical image segmentation foundation models across four (pre-)clinically relevant imaging modalities in zero-, one-, and few-shot scenarios, therefore providing a universal solution for 3D blood vessel segmentation.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is the challenges of 3D vessel segmentation in medical image analysis. Specifically, the 3D vessel segmentation task faces difficulties in the following aspects: 1. **Modality - specific variations**: Different imaging techniques (such as MRA, CTA, OCTA, etc.) can lead to significant imaging artifacts, variations in vessel patterns and scales, signal - to - noise ratios, and background tissues. 2. **Inter - domain differences**: Due to differences in imaging protocols, the domain gap between different datasets is large, which limits the generalization ability of existing supervised learning methods and requires cumbersome voxel - level annotation for each dataset. 3. **Lack of zero - shot generalization ability**: Although existing base models perform well on other tasks, they usually cannot generalize well to unseen data domains in the 3D vessel segmentation task. To solve these problems, the author proposes a base model named vesselFM, which is specifically designed for a wide range of 3D vessel segmentation tasks. vesselFM achieves zero - shot generalization ability by training on three heterogeneous data sources: - **Dreal**: A large, carefully annotated real - world dataset that covers multiple imaging modalities and multiple anatomical regions of different organisms. - **Ddrand**: A synthetic dataset generated by a domain randomization strategy, aiming to cover the general domain of 3D vascular images. - **Dflow**: High - fidelity image - mask pairs generated by flow matching, and these data follow consistent anatomical constraints. Through the combined use of these data sources, vesselFM can achieve zero - shot generalization on unseen data domains and exhibits performance superior to existing state - of - the - art models in zero - shot, one - shot, and few - shot scenarios.