Abstract:The fundamental motivation of the proposed work is to present a new visualization-guided computing paradigm to combine direct 3D volume processing and volume rendered clues for effective 3D exploration. For example, extracting and visualizing microstructures in-vivo have been a long-standing challenging problem. However, due to the high sparseness and noisiness in cerebrovasculature data as well as highly complex geometry and topology variations of micro vessels, it is still extremely challenging to extract the complete 3D vessel structure and visualize it in 3D with high fidelity. In this paper, we present an end-to-end deep learning method, VC-Net, for robust extraction of 3D microvascular structure through embedding the image composition, generated by maximum intensity projection (MIP), into the 3D volumetric image learning process to enhance the overall performance. The core novelty is to automatically leverage the volume visualization technique (e.g., MIP - a volume rendering scheme for 3D volume images) to enhance the 3D data exploration at the deep learning level. The MIP embedding features can enhance the local vessel signal (through canceling out the noise) and adapt to the geometric variability and scalability of vessels, which is of great importance in microvascular tracking. A multi-stream convolutional neural network (CNN) framework is proposed to effectively learn the 3D volume and 2D MIP feature vectors, respectively, and then explore their inter-dependencies in a joint volume-composition embedding space by unprojecting the 2D feature vectors into the 3D volume embedding space. It is noted that the proposed framework can better capture the small/micro vessels and improve the vessel connectivity. To our knowledge, this is the first time that a deep learning framework is proposed to construct a joint convolutional embedding space, where the computed vessel probabilities from volume rendering based 2D projection and 3D volume can be explored and integrated synergistically. Experimental results are evaluated and compared with the traditional 3D vessel segmentation methods and the state-of-the-art in deep learning, by using extensive public and real patient (micro- )cerebrovascular image datasets. The application of this accurate segmentation and visualization of sparse and complicated 3D microvascular structure facilitated by our method demonstrates the potential in a powerful MR arteriogram and venogram diagnosis of vascular disease.

Cascaded 3D Object Segmentation with Volumetric Propagation Network

3D Multiple-Contextual ROI-Attention Network for Efficient and Accurate Volumetric Medical Image Segmentation.

Adaptive Decomposition and Shared Weight Volumetric Transformer Blocks for Efficient Patch-Free 3D Medical Image Segmentation.

A 3D Convolutional Neural Network for Volumetric Image Semantic Segmentation

Graph Convolution Based Cross-Network Multiscale Feature Fusion for Deep Vessel Segmentation

Deep Sequential Segmentation of Organs in Volumetric Medical Scans

Atrous Residual Interconnected Encoder to Attention Decoder Framework for Vertebrae Segmentation via 3D Volumetric CT Images

VC-Net: Deep Volume-Composition Networks for Segmentation and Visualization of Highly Sparse and Noisy Image Data

An application of cascaded 3D fully convolutional networks for medical image segmentation

Automatic 3D Medical Image Segmentation with Densely-Connected Volumetric ConvNets

3D LVCN: A Lightweight Volumetric ConvNet

Bridging 2D and 3D Segmentation Networks for Computation Efficient Volumetric Medical Image Segmentation: An Empirical Study of 2.5D Solutions

A 3D Coarse-to-Fine Framework for Volumetric Medical Image Segmentation

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

DVNet: A Memory-Efficient Three-Dimensional CNN for Large-Scale Neurovascular Reconstruction

Voxel-Based 3D Shape Segmentation Using Deep Volumetric Convolutional Neural Networks

PnP-AE: A Plug-and-Play Module for Volumetric Medical Image Segmentation

Convolutional Neural Network with Asymmetric Encoding and Decoding Structure for Brain Vessel Segmentation on Computed Tomographic Angiography.

Segmentation and Vascular Vectorization for Coronary Artery by Geometry-based Cascaded Neural Network

Med-DANet: Dynamic Architecture Network for Efficient Medical Volumetric Segmentation

V-NAS: Neural Architecture Search for Volumetric Medical Image Segmentation