Abstract:The fundamental motivation of the proposed work is to present a new visualization-guided computing paradigm to combine direct 3D volume processing and volume rendered clues for effective 3D exploration. For example, extracting and visualizing microstructures in-vivo have been a long-standing challenging problem. However, due to the high sparseness and noisiness in cerebrovasculature data as well as highly complex geometry and topology variations of micro vessels, it is still extremely challenging to extract the complete 3D vessel structure and visualize it in 3D with high fidelity. In this paper, we present an end-to-end deep learning method, VC-Net, for robust extraction of 3D microvascular structure through embedding the image composition, generated by maximum intensity projection (MIP), into the 3D volumetric image learning process to enhance the overall performance. The core novelty is to automatically leverage the volume visualization technique (e.g., MIP - a volume rendering scheme for 3D volume images) to enhance the 3D data exploration at the deep learning level. The MIP embedding features can enhance the local vessel signal (through canceling out the noise) and adapt to the geometric variability and scalability of vessels, which is of great importance in microvascular tracking. A multi-stream convolutional neural network (CNN) framework is proposed to effectively learn the 3D volume and 2D MIP feature vectors, respectively, and then explore their inter-dependencies in a joint volume-composition embedding space by unprojecting the 2D feature vectors into the 3D volume embedding space. It is noted that the proposed framework can better capture the small/micro vessels and improve the vessel connectivity. To our knowledge, this is the first time that a deep learning framework is proposed to construct a joint convolutional embedding space, where the computed vessel probabilities from volume rendering based 2D projection and 3D volume can be explored and integrated synergistically. Experimental results are evaluated and compared with the traditional 3D vessel segmentation methods and the state-of-the-art in deep learning, by using extensive public and real patient (micro- )cerebrovascular image datasets. The application of this accurate segmentation and visualization of sparse and complicated 3D microvascular structure facilitated by our method demonstrates the potential in a powerful MR arteriogram and venogram diagnosis of vascular disease.

Semi-overcomplete convolutional auto-encoder embedding as shape priors for deep vessel segmentation

Deep vessel segmentation with joint multi-prior encoding

Accurate Retinal Vessel Segmentation via Octave Convolution Neural Network

VesselNet: A deep convolutional neural network with multi pathways for robust hepatic vessel segmentation

Graph Convolution Based Cross-Network Multiscale Feature Fusion for Deep Vessel Segmentation

Deep vessel segmentation based on a new combination of vesselness filters

Deep Retinal Image Segmentation with Regularization Under Geometric Priors

Deep Dilated Convolutional Nets for the Automatic Segmentation of Retinal Vessels

JointVesselNet: Joint Volume-Projection Convolutional Embedding Networks for 3D Cerebrovascular Segmentation

Orientation and Context Entangled Network for Retinal Vessel Segmentation

CAVE: Cerebral Artery-Vein Segmentation in Digital Subtraction Angiography

Sequential Vessel Segmentation Via Deep Channel Attention Network

A U-Net Deep Learning Framework for High Performance Vessel Segmentation in Patients With Cerebrovascular Disease

Deep Neural Ensemble for Retinal Vessel Segmentation in Fundus Images towards Achieving Label-free Angiography

VC-Net: Deep Volume-Composition Networks for Segmentation and Visualization of Highly Sparse and Noisy Image Data

Multi-Scale Retina Vessel Segmentation in OCTA with a Vascular Connectivity Module in the Convolutional Neural Network.

Dual-path multi-scale context dense aggregation network for retinal vessel segmentation

Rethinking the Extraction and Interaction of Multi-Scale Features for Vessel Segmentation

Retinal vessel image segmentation algorithm based on encoder-decoder structure

Contextual Information Enhanced Convolutional Neural Networks for Retinal Vessel Segmentation in Color Fundus Images

Optimizing ensemble U-Net architectures for robust coronary vessel segmentation in angiographic images