Abstract:Prominent solutions for medical image segmentation are typically tailored for automatic or interactive setups, posing challenges in facilitating progress achieved in one task to another.$_{\!}$ This$_{\!}$ also$_{\!}$ necessitates$_{\!}$ separate$_{\!}$ models for each task, duplicating both training time and parameters.$_{\!}$ To$_{\!}$ address$_{\!}$ above$_{\!}$ issues,$_{\!}$ we$_{\!}$ introduce$_{\!}$ S2VNet,$_{\!}$ a$_{\!}$ universal$_{\!}$ framework$_{\!}$ that$_{\!}$ leverages$_{\!}$ Slice-to-Volume$_{\!}$ propagation$_{\!}$ to$_{\!}$ unify automatic/interactive segmentation within a single model and one training session. Inspired by clustering-based segmentation techniques, S2VNet makes full use of the slice-wise structure of volumetric data by initializing cluster centers from the cluster$_{\!}$ results$_{\!}$ of$_{\!}$ previous$_{\!}$ slice.$_{\!}$ This enables knowledge acquired from prior slices to assist in the segmentation of the current slice, further efficiently bridging the communication between remote slices using mere 2D networks. Moreover, such a framework readily accommodates interactive segmentation with no architectural change, simply by initializing centroids from user inputs. S2VNet distinguishes itself by swift inference speeds and reduced memory consumption compared to prevailing 3D solutions. It can also handle multi-class interactions with each of them serving to initialize different centroids. Experiments on three benchmarks demonstrate S2VNet surpasses task-specified solutions on both automatic/interactive setups.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address several key issues in the field of medical image segmentation: 1. **Limitations of Task-Specific Models**: - Current medical image segmentation solutions are typically customized for either automatic or interactive settings, making it difficult for progress in one task to benefit others. - Different models need to be trained separately for each task, increasing training time and the number of parameters. 2. **Efficiency Issues of 3D Networks**: - Existing volumetric segmentation methods rely on 3D networks, which are slow in inference speed and face significant challenges when deployed on hospital equipment due to limited computational power. - Using sliding window inference to handle large memory consumption further hinders the propagation of user input throughout the volume. 3. **Limitations of Multi-Class Interaction**: - Current interactive solutions typically handle only a single foreground class, while automatic methods have made significant progress in multi-class segmentation. ### Solution To overcome the above issues, the authors propose S2VNet (Slice-to-Volume Network), a universal segmentation framework that achieves its goals through the following means: 1. **Unifying Automatic and Interactive Segmentation**: - S2VNet can handle both automatic and interactive segmentation tasks within a single model and a single training session. - By initializing cluster centers from user input, S2VNet can seamlessly adapt to interactive segmentation without modifying the network architecture. 2. **Efficient 2D Network**: - Utilizing a 2D network with a slice-to-volume propagation mechanism, S2VNet can efficiently bridge distant slices, significantly reducing computational resource consumption while maintaining segmentation accuracy (e.g., 15 times faster inference speed and 48.2% less memory consumption compared to 3D networks). 3. **Multi-Class Interaction Support**: - S2VNet can accept user inputs for multiple classes simultaneously, initializing a cluster center for each class, and optimizing different classes of multiple instances in parallel within a single network forward pass. 4. **Robust Inference Mechanism**: - Through a recursive cluster center aggregation strategy, S2VNet can collect historical cluster centers and fuse them into a vector, maintaining persistent awareness of distant structural cues during propagation. ### Experimental Validation The authors conducted experiments on three benchmark datasets (WORD, BTCV, AMOS), and the results demonstrate that S2VNet performs excellently in both automatic and interactive segmentation tasks, even surpassing solutions specifically designed for these tasks. In summary, S2VNet provides a universal and efficient medical image segmentation framework through an innovative slice-to-volume propagation mechanism, significantly enhancing the performance of both automatic and interactive segmentation.

Clustering Propagation for Universal Medical Image Segmentation

Adaptive Decomposition and Shared Weight Volumetric Transformer Blocks for Efficient Patch-Free 3D Medical Image Segmentation.

3D U$$^$$-Net: A 3D Universal U-Net for Multi-domain Medical Image Segmentation

A novel medical image segmentation approach by using multi-branch segmentation network based on local and global information synchronous learning

One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation

SegVol: Universal and Interactive Volumetric Medical Image Segmentation

PropSAM: A Propagation-Based Model for Segmenting Any 3D Objects in Multi-Modal Medical Images

M$^{2}$SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation

PAM: A Propagation-Based Model for Segmenting Any 3D Objects across Multi-Modal Medical Images

Sli2Vol+: Segmenting 3D Medical Images Based on an Object Estimation Guided Correspondence Flow Network

VCMix-Net: A hybrid network for medical image segmentation

Volumetric memory network for interactive medical image segmentation

SBDF-Net: A Versatile Dual-Branch Fusion Network for Medical Image Segmentation

PIS-Net: Efficient Medical Image Segmentation Network with Multivariate Downsampling for Point-of-Care

Multi-view Hierarchical Split Network for Brain Tumor Segmentation.

Ultra-Lightweight Network for Medical Image Segmentation Inspired by Bio-Visual Interaction

Bridging 2D and 3D Segmentation Networks for Computation Efficient Volumetric Medical Image Segmentation: An Empirical Study of 2.5D Solutions

ShiftTransUNet: An Efficient Deep Learning Model for Medical Image Segmentation Using ShiftViT Framework

A Unified 2D Medical Image Segmentation Network (segmentnet) Through Distance-Awareness and Local Feature Extraction

Make U-Net Greater: An Easy-to-Embed Approach to Improve Segmentation Performance Using Hypergraph

H-vmunet: High-order Vision Mamba UNet for Medical Image Segmentation