Abstract:Convolutional neural networks (CNNs) are the current state-of-the-art meta-algorithm for volumetric segmentation of medical data, for example, to localize COVID-19 infected tissue on computer tomography scans or the detection of tumour volumes in magnetic resonance imaging. A key limitation of 3D CNNs on voxelised data is that the memory consumption grows cubically with the training data resolution. Occupancy networks (O-Nets) are an alternative for which the data is represented continuously in a function space and 3D shapes are learned as a continuous decision boundary. While O-Nets are significantly more memory efficient than 3D CNNs, they are limited to simple shapes, are relatively slow at inference, and have not yet been adapted for 3D semantic segmentation of medical data. Here, we propose Occupancy Networks for Semantic Segmentation (OSS-Nets) to accurately and memory-efficiently segment 3D medical data. We build upon the original O-Net with modifications for increased expressiveness leading to improved segmentation performance comparable to 3D CNNs, as well as modifications for faster inference. We leverage local observations to represent complex shapes and prior encoder predictions to expedite inference. We showcase OSS-Net's performance on 3D brain tumour and liver segmentation against a function space baseline (O-Net), a performance baseline (3D residual U-Net), and an efficiency baseline (2D residual U-Net). OSS-Net yields segmentation results similar to the performance baseline and superior to the function space and efficiency baselines. In terms of memory efficiency, OSS-Net consumes comparable amounts of memory as the function space baseline, somewhat more memory than the efficiency baseline and significantly less than the performance baseline. As such, OSS-Net enables memory-efficient and accurate 3D semantic segmentation that can scale to high resolutions.

Random 2.5D U-net for Fully 3D Segmentation

3D Multiple-Contextual ROI-Attention Network for Efficient and Accurate Volumetric Medical Image Segmentation.

DeU-Net: Deformable U-Net for 3D Cardiac MRI Video Segmentation

DeU-Net 2.0: Enhanced deformable U-Net for 3D cardiac cine MRI segmentation

3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation

3D U$$^$$-Net: A 3D Universal U-Net for Multi-domain Medical Image Segmentation

One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation

A Partially Reversible U-Net for Memory-Efficient Volumetric Image Segmentation

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Dilated Nearest-Neighbor Encoding for 3D Semantic Segmentation of Point Clouds

A two-stage 3D Unet framework for multi-class segmentation on full resolution image

Deep Sequential Segmentation of Organs in Volumetric Medical Scans

Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation

Attention 3D U-Net with Multiple Skip Connections for Segmentation of Brain Tumor Images

2.5d Convolution For Rgb-D Semantic Segmentation

Learning to Segment 3D Point Clouds in 2D Image Space

3D Global Convolutional Adversarial Network\\ for Prostate MR Volume Segmentation

3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation

OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data

Comparison of 2D vs. 3D U-Net Organ Segmentation in abdominal 3D CT images

Recurrent U-Net for Resource-Constrained Segmentation