Long-term results of stenting of the aortic bifurcation.

N. Abello,B. Kretz,J. Picquet,P. Magnan,R. Hassen-Khodja,J. Chevalier,E. Rosset,P. Feugier,M. Fleury,E. Steinmetz

DOI: https://doi.org/10.1016/j.avsg.2011.05.046

IF: 1.5

2012-05-01

Annals of Vascular Surgery

Abstract:

What problem does this paper attempt to address?

3D Deep Learning on Medical Images: A Review

Satya P. Singh,Lipo Wang,Sukrit Gupta,Haveesh Goli,Parasuraman Padmanabhan,Balázs Gulyás

DOI: https://doi.org/10.48550/arXiv.2004.00218

2020-10-13

Abstract:The rapid advancements in machine learning, graphics processing technologies and the availability of medical imaging data have led to a rapid increase in the use of deep learning models in the medical domain. This was exacerbated by the rapid advancements in convolutional neural network (CNN) based architectures, which were adopted by the medical imaging community to assist clinicians in disease diagnosis. Since the grand success of AlexNet in 2012, CNNs have been increasingly used in medical image analysis to improve the efficiency of human clinicians. In recent years, three-dimensional (3D) CNNs have been employed for the analysis of medical images. In this paper, we trace the history of how the 3D CNN was developed from its machine learning roots, we provide a brief mathematical description of 3D CNN and provide the preprocessing steps required for medical images before feeding them to 3D CNNs. We review the significant research in the field of 3D medical imaging analysis using 3D CNNs (and its variants) in different medical areas such as classification, segmentation, detection and localization. We conclude by discussing the challenges associated with the use of 3D CNNs in the medical imaging domain (and the use of deep learning models in general) and possible future trends in the field.

Quantitative Methods,Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
Super Images -- A New 2D Perspective on 3D Medical Imaging Analysis

Ikboljon Sobirov,Numan Saeed,Mohammad Yaqub

DOI: https://doi.org/10.48550/arXiv.2205.02847

2023-05-17

Abstract:In medical imaging analysis, deep learning has shown promising results. We frequently rely on volumetric data to segment medical images, necessitating the use of 3D architectures, which are commended for their capacity to capture interslice context. However, because of the 3D convolutions, max pooling, up-convolutions, and other operations utilized in these networks, these architectures are often more inefficient in terms of time and computation than their 2D equivalents. Furthermore, there are few 3D pretrained model weights, and pretraining is often difficult. We present a simple yet effective 2D method to handle 3D data while efficiently embedding the 3D knowledge during training. We propose transforming volumetric data into 2D super images and segmenting with 2D networks to solve these challenges. Our method generates a super-resolution image by stitching slices side by side in the 3D image. We expect deep neural networks to capture and learn these properties spatially despite losing depth information. This work aims to present a novel perspective when dealing with volumetric data, and we test the hypothesis using CNN and ViT networks as well as self-supervised pretraining. While attaining equal, if not superior, results to 3D networks utilizing only 2D counterparts, the model complexity is reduced by around threefold. Because volumetric data is relatively scarce, we anticipate that our approach will entice more studies, particularly in medical imaging analysis.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition
Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data

Ivan Diaz,Mario Geiger,Richard Iain McKinley

DOI: https://doi.org/10.59275/j.melba.2024-7189

2024-05-17

Abstract:Convolutional neural networks (CNNs) allow for parameter sharing and translational equivariance by using convolutional kernels in their linear layers. By restricting these kernels to be SO(3)-steerable, CNNs can further improve parameter sharing. These rotationally-equivariant convolutional layers have several advantages over standard convolutional layers, including increased robustness to unseen poses, smaller network size, and improved sample efficiency. Despite this, most segmentation networks used in medical image analysis continue to rely on standard convolutional kernels. In this paper, we present a new family of segmentation networks that use equivariant voxel convolutions based on spherical harmonics. These networks are robust to data poses not seen during training, and do not require rotation-based data augmentation during training. In addition, we demonstrate improved segmentation performance in MRI brain tumor and healthy brain structure segmentation tasks, with enhanced robustness to reduced amounts of training data and improved parameter efficiency. Code to reproduce our results, and to implement the equivariant segmentation networks for other tasks is available at

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
Cross-dimensional transfer learning in medical image segmentation with deep learning

Hicham Messaoudi,Ahror Belaid,Douraied Ben Salem,Pierre-Henri Conze

DOI: https://doi.org/10.1016/j.media.2023.102868

2023-07-29

Abstract:Over the last decade, convolutional neural networks have emerged and advanced the state-of-the-art in various image analysis and computer vision applications. The performance of 2D image classification networks is constantly improving and being trained on databases made of millions of natural images. However, progress in medical image analysis has been hindered by limited annotated data and acquisition constraints. These limitations are even more pronounced given the volumetry of medical imaging data. In this paper, we introduce an efficient way to transfer the efficiency of a 2D classification network trained on natural images to 2D, 3D uni- and multi-modal medical image segmentation applications. In this direction, we designed novel architectures based on two key principles: weight transfer by embedding a 2D pre-trained encoder into a higher dimensional U-Net, and dimensional transfer by expanding a 2D segmentation network into a higher dimension one. The proposed networks were tested on benchmarks comprising different modalities: MR, CT, and ultrasound images. Our 2D network ranked first on the CAMUS challenge dedicated to echo-cardiographic data segmentation and surpassed the state-of-the-art. Regarding 2D/3D MR and CT abdominal images from the CHAOS challenge, our approach largely outperformed the other 2D-based methods described in the challenge paper on Dice, RAVD, ASSD, and MSSD scores and ranked third on the online evaluation platform. Our 3D network applied to the BraTS 2022 competition also achieved promising results, reaching an average Dice score of 91.69% (91.22%) for the whole tumor, 83.23% (84.77%) for the tumor core, and 81.75% (83.88%) for enhanced tumor using the approach based on weight (dimensional) transfer. Experimental and qualitative results illustrate the effectiveness of our methods for multi-dimensional medical image segmentation.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data

Xin Wang,Ruisheng Su,Weiyi Xie,Wenjin Wang,Yi Xu,Ritse Mann,Jungong Han,Tao Tan

DOI: https://doi.org/10.1016/j.bspc.2023.104858

2024-01-23

Abstract:In medical-data driven learning, 3D convolutional neural networks (CNNs) have started to show superior performance to 2D CNNs in numerous deep learning tasks, proving the added value of 3D spatial information in feature representation. However, the difficulty in collecting more training samples to converge, more computational resources and longer execution time make this approach less applied. Also, applying transfer learning on 3D CNN is challenging due to a lack of publicly available pre-trained 3D models. To tackle these issues, we proposed a novel 2D strategical representation of volumetric data, namely 2.75D. In this work, the spatial information of 3D images is captured in a single 2D view by a spiral-spinning technique. As a result, 2D CNN networks can also be used to learn volumetric information. Besides, we can fully leverage pre-trained 2D CNNs for downstream vision problems. We also explore a multi-view 2.75D strategy, 2.75D 3 channels (2.75Dx3), to boost the advantage of 2.75D. We evaluated the proposed methods on three public datasets with different modalities or organs (Lung CT, Breast MRI, and Prostate MRI), against their 2D, 2.5D, and 3D counterparts in classification tasks. Results show that the proposed methods significantly outperform other counterparts when all methods were trained from scratch on the lung dataset. Such performance gain is more pronounced with transfer learning or in the case of limited training data. Our methods also achieved comparable performance on other datasets. In addition, our methods achieved a substantial reduction in time consumption of training and inference compared with the 2.5D or 3D method.

Image and Video Processing,Computer Vision and Pattern Recognition
3-D Convolutional Neural Networks for Glioblastoma Segmentation

Darvin Yi,Mu Zhou,Zhao Chen,Olivier Gevaert

DOI: https://doi.org/10.48550/arXiv.1611.04534

2016-11-15

Abstract:Convolutional Neural Networks (CNN) have emerged as powerful tools for learning discriminative image features. In this paper, we propose a framework of 3-D fully CNN models for Glioblastoma segmentation from multi-modality MRI data. By generalizing CNN models to true 3-D convolutions in learning 3-D tumor MRI data, the proposed approach utilizes a unique network architecture to decouple image pixels. Specifically, we design a convolutional layer with pre-defined Difference- of-Gaussian (DoG) filters to perform true 3-D convolution incorporating local neighborhood information at each pixel. We then use three trained convolutional layers that act to decouple voxels from the initial 3-D convolution. The proposed framework allows identification of high-level tumor structures on MRI. We evaluate segmentation performance on the BRATS segmentation dataset with 274 tumor samples. Extensive experimental results demonstrate encouraging performance of the proposed approach comparing to the state-of-the-art methods. Our data-driven approach achieves a median Dice score accuracy of 89% in whole tumor glioblastoma segmentation, revealing a generalized low-bias possibility to learn from medium-size MRI datasets.

Computer Vision and Pattern Recognition
Performance of a Deep Neural Network Algorithm Based on a Small Medical Image Dataset: Incremental Impact of 3D-to-2D Reformation Combined with Novel Data Augmentation, Photometric Conversion, or Transfer Learning

Vikash Gupta,Mutlu Demirer,Matthew Bigelow,Kevin J. Little,Sema Candemir,Luciano M. Prevedello,Richard D. White,Thomas P. O’Donnell,Michael Wels,Barbaros S. Erdal

DOI: https://doi.org/10.1007/s10278-019-00267-3

IF: 4.903

2019-10-17

Journal of Digital Imaging

Abstract:Collecting and curating large medical-image datasets for deep neural network (DNN) algorithm development is typically difficult and resource-intensive. While transfer learning (TL) decreases reliance on large data collections, current TL implementations are tailored to two-dimensional (2D) datasets, limiting applicability to volumetric imaging (e.g., computed tomography). Targeting performance enhancement of a DNN algorithm based on a small image dataset, we assessed incremental impact of 3D-to-2D projection methods, one supporting novel data augmentation (DA); photometric grayscale-to-color conversion (GCC); and/or TL on training of an algorithm from a small coronary computed tomography angiography (CCTA) dataset (200 examinations, 50% with atherosclerosis and 50% atherosclerosis-free) producing 245 diseased and 1127 normal coronary arteries/branches. Volumetric CCTA data was converted to a 2D format creating both an Aggregate Projection View (APV) and a Mosaic Projection View (MPV), supporting DA per vessel; both grayscale and color-mapped versions of each view were also obtained. Training was performed both without and with TL, and algorithm performance of all permutations was compared using area under the receiver operating characteristics curve. Without TL, APV performance was 0.74 and 0.87 on grayscale and color images, respectively, compared to 0.90 and 0.87 for MPV. With TL, APV performance was 0.78 and 0.88 on grayscale and color images, respectively, compared with 0.93 and 0.91 for MPV. In conclusion, TL enhances performance of a DNN algorithm from a small volumetric dataset after proposed 3D-to-2D reformatting, but additive gain is achieved with application of either GCC to APV or the proposed novel MPV technique for DA.

radiology, nuclear medicine & medical imaging
One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation

Mathias Perslev,Erik Bjørnager Dam,Akshay Pai,Christian Igel

DOI: https://doi.org/10.48550/arXiv.1911.01764

IF: 5.414

2019-11-05

Machine Learning

Abstract:Many recent medical segmentation systems rely on powerful deep learning models to solve highly specific tasks. To maximize performance, it is standard practice to evaluate numerous pipelines with varying model topologies, optimization parameters, pre- & postprocessing steps, and even model cascades. It is often not clear how the resulting pipeline transfers to different tasks. We propose a simple and thoroughly evaluated deep learning framework for segmentation of arbitrary medical image volumes. The system requires no task-specific information, no human interaction and is based on a fixed model topology and a fixed hyperparameter set, eliminating the process of model selection and its inherent tendency to cause method-level over-fitting. The system is available in open source and does not require deep learning expertise to use. Without task-specific modifications, the system performed better than or similar to highly specialized deep learning methods across 3 separate segmentation tasks. In addition, it ranked 5-th and 6-th in the first and second round of the 2018 Medical Segmentation Decathlon comprising another 10 tasks. The system relies on multi-planar data augmentation which facilitates the application of a single 2D architecture based on the familiar U-Net. Multi-planar training combines the parameter efficiency of a 2D fully convolutional neural network with a systematic train- and test-time augmentation scheme, which allows the 2D model to learn a representation of the 3D image volume that fosters generalization.
A low resource 3D U-Net based deep learning model for medical image analysis

Girija Chetty,Mohammad Yamin,Matthew White

DOI: https://doi.org/10.1007/s41870-021-00850-4

2022-01-05

International Journal of Information Technology

Abstract:The success of deep learning, a subfield of Artificial Intelligence technologies in the field of image analysis and computer can be leveraged for building better decision support systems for clinical radiological settings. Detecting and segmenting tumorous tissues in brain region using deep learning and artificial intelligence is one such scenario, where radiologists can benefit from the computer based second opinion or decision support, for detecting the severity of disease, and survival of the subject with an accurate and timely clinical diagnosis. Gliomas are the aggressive form of brain tumors having irregular shape and ambiguous boundaries, making them one of the hardest tumors to detect, and often require a combined analysis of different types of radiological scans to make an accurate detection. In this paper, we present a fully automatic deep learning method for brain tumor segmentation in multimodal multi-contrast magnetic resonance image scans. The proposed approach is based on light weight UNET architecture, consisting of a multimodal CNN encoder-decoder based computational model. Using the publicly available Brain Tumor Segmentation (BraTS) Challenge 2018 dataset, available from the Medical Image Computing and Computer Assisted Intervention (MICCAI) society, our novel approach based on proposed light-weight UNet model, with no data augmentation requirements and without use of heavy computational resources, has resulted in an improved performance, as compared to the previous models in the challenge task that used heavy computational architectures and resources and with different data augmentation approaches. This makes the model proposed in this work more suitable for remote, extreme and low resource health care settings.
CNN-based Segmentation of Medical Imaging Data

Baris Kayalibay,Grady Jensen,Patrick van der Smagt

DOI: https://doi.org/10.48550/arXiv.1701.03056

2017-07-25

Abstract:Convolutional neural networks have been applied to a wide variety of computer vision tasks. Recent advances in semantic segmentation have enabled their application to medical image segmentation. While most CNNs use two-dimensional kernels, recent CNN-based publications on medical image segmentation featured three-dimensional kernels, allowing full access to the three-dimensional structure of medical images. Though closely related to semantic segmentation, medical image segmentation includes specific challenges that need to be addressed, such as the scarcity of labelled data, the high class imbalance found in the ground truth and the high memory demand of three-dimensional images. In this work, a CNN-based method with three-dimensional filters is demonstrated and applied to hand and brain MRI. Two modifications to an existing CNN architecture are discussed, along with methods on addressing the aforementioned challenges. While most of the existing literature on medical image segmentation focuses on soft tissue and the major organs, this work is validated on data both from the central nervous system as well as the bones of the hand.

Computer Vision and Pattern Recognition
3D Self-Supervised Methods for Medical Imaging

Aiham Taleb,Winfried Loetzsch,Noel Danz,Julius Severin,Thomas Gaertner,Benjamin Bergner,Christoph Lippert

DOI: https://doi.org/10.48550/arXiv.2006.03829

2020-06-06

Computer Vision and Pattern Recognition

Abstract:Self-supervised learning methods have witnessed a recent surge of interest after proving successful in multiple application fields. In this work, we leverage these techniques, and we propose 3D versions for five different self-supervised methods, in the form of proxy tasks. Our methods facilitate neural network feature learning from unlabeled 3D images, aiming to reduce the required cost for expert annotation. The developed algorithms are 3D Contrastive Predictive Coding, 3D Rotation prediction, 3D Jigsaw puzzles, Relative 3D patch location, and 3D Exemplar networks. Our experiments show that pretraining models with our 3D tasks yields more powerful semantic representations, and enables solving downstream tasks more accurately and efficiently, compared to training the models from scratch and to pretraining them on 2D slices. We demonstrate the effectiveness of our methods on three downstream tasks from the medical imaging domain: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segmentation from 3D CT, and iii) Diabetic Retinopathy Detection from 2D Fundus images. In each task, we assess the gains in data-efficiency, performance, and speed of convergence. Interestingly, we also find gains when transferring the learned representations, by our methods, from a large unlabeled 3D corpus to a small downstream-specific dataset. We achieve results competitive to state-of-the-art solutions at a fraction of the computational expense. We publish our implementations for the developed algorithms (both 3D and 2D versions) as an open-source library, in an effort to allow other researchers to apply and extend our methods on their datasets.
Pretrained Deep 2.5D Models for Efficient Predictive Modeling from Retinal OCT

Taha Emre,Marzieh Oghbaie,Arunava Chakravarty,Antoine Rivail,Sophie Riedl,Julia Mai,Hendrik P.N. Scholl,Sobha Sivaprasad,Daniel Rueckert,Andrew Lotery,Ursula Schmidt-Erfurth,Hrvoje Bogunović

DOI: https://doi.org/10.1007/978-3-031-44013-7_14

2023-07-26

Abstract:In the field of medical imaging, 3D deep learning models play a crucial role in building powerful predictive models of disease progression. However, the size of these models presents significant challenges, both in terms of computational resources and data requirements. Moreover, achieving high-quality pretraining of 3D models proves to be even more challenging. To address these issues, hybrid 2.5D approaches provide an effective solution for utilizing 3D volumetric data efficiently using 2D models. Combining 2D and 3D techniques offers a promising avenue for optimizing performance while minimizing memory requirements. In this paper, we explore 2.5D architectures based on a combination of convolutional neural networks (CNNs), long short-term memory (LSTM), and Transformers. In addition, leveraging the benefits of recent non-contrastive pretraining approaches in 2D, we enhanced the performance and data efficiency of 2.5D techniques even further. We demonstrate the effectiveness of architectures and associated pretraining on a task of predicting progression to wet age-related macular degeneration (AMD) within a six-month period on two large longitudinal OCT datasets.

Computer Vision and Pattern Recognition,Machine Learning
Leveraging Convolutional Neural Networks for 3D Quantitative Angiography Reconstructions from Sparse Cone Beam CT Projections Utilizing CFD Data

Ahmad Rahmatpour,Allison Shields,Parmita Mondal,Parisa Naghdi,Michael Udin,Kyle A Williams,Mohammad Mahdi Shiraz Bhurwani,Swetadri Vasan Setlur Nagesh,Ciprian N Ionita

2024-11-21

Abstract:This study leverages convolutional neural networks to enhance the temporal resolution of 3D angiography in intracranial aneurysms focusing on the reconstruction of volumetric contrast data from sparse and limited projections. Three patient-specific IA geometries were segmented and converted into stereolithography files to facilitate computational fluid dynamics simulations. These simulations first modeled blood flow under steady conditions with varying inlet velocities: 0.25 m/s, 0.35 m/s, and 0.45 m/s. Subsequently, 3D angiograms were simulated by labeling inlet particles to represent contrast bolus injections over durations of 0.5s, 1.0s, 1.5s, and 2.0s. The angiographic simulations were then used within a simulated cone beam C arm CT system to generate in-silico rotational DSAs, capturing projections every 10 ms over a 220-degree arc at 27 frames per second. From these simulations, both fully sampled (108 projections) and truncated projection datasets were generated the latter using a maximum of 49 projections. High fidelity volumetric images were reconstructed using a Parker weighted Feldkamp Davis Kress algorithm. A modified U Net CNN was subsequently trained on these datasets to reconstruct 3D angiographic volumes from the truncated projections. The network incorporated multiple convolutional layers with ReLU activations and Max pooling, complemented by upsampling and concatenation to preserve spatial detail. Model performance was evaluated using mean squared error (MSE). Evaluating our U net model across the test set yielded a MSE of 0.0001, indicating good agreement with ground truth reconstructions and demonstrating acceptable capabilities in capturing relevant transient angiographic features. This study confirms the feasibility of using CNNs for reconstructing 3D angiographic images from truncated projections.

Medical Physics
Transferring Models Trained on Natural Images to 3D MRI via Position Encoded Slice Models

Umang Gupta,Tamoghna Chattopadhyay,Nikhil Dhinagar,Paul M. Thompson,Greg Ver Steeg,Alzheimer's Disease Neuroimaging Initiative

DOI: https://doi.org/10.48550/arXiv.2303.01491

2023-03-03

Abstract:Transfer learning has remarkably improved computer vision. These advances also promise improvements in neuroimaging, where training set sizes are often small. However, various difficulties arise in directly applying models pretrained on natural images to radiologic images, such as MRIs. In particular, a mismatch in the input space (2D images vs. 3D MRIs) restricts the direct transfer of models, often forcing us to consider only a few MRI slices as input. To this end, we leverage the 2D-Slice-CNN architecture of Gupta et al. (2021), which embeds all the MRI slices with 2D encoders (neural networks that take 2D image input) and combines them via permutation-invariant layers. With the insight that the pretrained model can serve as the 2D encoder, we initialize the 2D encoder with ImageNet pretrained weights that outperform those initialized and trained from scratch on two neuroimaging tasks -- brain age prediction on the UK Biobank dataset and Alzheimer's disease detection on the ADNI dataset. Further, we improve the modeling capabilities of 2D-Slice models by incorporating spatial information through position embeddings, which can improve the performance in some cases.

Image and Video Processing,Machine Learning,Quantitative Methods
Spatiotemporal Modeling Encounters 3D Medical Image Analysis: Slice-Shift UNet with Multi-View Fusion

C. I. Ugwu,S. Casarin,O. Lanz

2023-07-25

Abstract:As a fundamental part of computational healthcare, Computer Tomography (CT) and Magnetic Resonance Imaging (MRI) provide volumetric data, making the development of algorithms for 3D image analysis a necessity. Despite being computationally cheap, 2D Convolutional Neural Networks can only extract spatial information. In contrast, 3D CNNs can extract three-dimensional features, but they have higher computational costs and latency, which is a limitation for clinical practice that requires fast and efficient models. Inspired by the field of video action recognition we propose a new 2D-based model dubbed Slice SHift UNet (SSH-UNet) which encodes three-dimensional features at 2D CNN's complexity. More precisely multi-view features are collaboratively learned by performing 2D convolutions along the three orthogonal planes of a volume and imposing a weights-sharing mechanism. The third dimension, which is neglected by the 2D convolution, is reincorporated by shifting a portion of the feature maps along the slices' axis. The effectiveness of our approach is validated in Multi-Modality Abdominal Multi-Organ Segmentation (AMOS) and Multi-Atlas Labeling Beyond the Cranial Vault (BTCV) datasets, showing that SSH-UNet is more efficient while on par in performance with state-of-the-art architectures.

Image and Video Processing,Computer Vision and Pattern Recognition
Comparative Evaluation of 3D and 2D Deep Learning Techniques for Semantic Segmentation in CT Scans

Abhishek Shivdeo,Rohit Lokwani,Viraj Kulkarni,Amit Kharat,Aniruddha Pant

DOI: https://doi.org/10.48550/arXiv.2101.07612

2021-01-19

Abstract:Image segmentation plays a pivotal role in several medical-imaging applications by assisting the segmentation of the regions of interest. Deep learning-based approaches have been widely adopted for semantic segmentation of medical data. In recent years, in addition to 2D deep learning architectures, 3D architectures have been employed as the predictive algorithms for 3D medical image data. In this paper, we propose a 3D stack-based deep learning technique for segmenting manifestations of consolidation and ground-glass opacities in 3D Computed Tomography (CT) scans. We also present a comparison based on the segmentation results, the contextual information retained, and the inference time between this 3D technique and a traditional 2D deep learning technique. We also define the area-plot, which represents the peculiar pattern observed in the slice-wise areas of the pathology regions predicted by these deep learning models. In our exhaustive evaluation, 3D technique performs better than the 2D technique for the segmentation of CT scans. We get dice scores of 79% and 73% for the 3D and the 2D techniques respectively. The 3D technique results in a 5X reduction in the inference time compared to the 2D technique. Results also show that the area-plots predicted by the 3D model are more similar to the ground truth than those predicted by the 2D model. We also show how increasing the amount of contextual information retained during the training can improve the 3D model's performance.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
An application of cascaded 3D fully convolutional networks for medical image segmentation

Holger R Roth,Hirohisa Oda,Xiangrong Zhou,Natsuki Shimizu,Ying Yang,Yuichiro Hayashi,Masahiro Oda,Michitaka Fujiwara,Kazunari Misawa,Kensaku Mori,Holger R. Roth

DOI: https://doi.org/10.1016/j.compmedimag.2018.03.001

IF: 7.422

2018-06-01

Computerized Medical Imaging and Graphics

Abstract:Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of several anatomical structures (ranging from the large organs to thin vessels) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training class-specific models. To this end, we propose a two-stage, coarse-to-fine approach that will first use a 3D FCN to roughly define a candidate region, which will then be used as input to a second 3D FCN. This reduces the number of voxels the second FCN has to classify to ∼10% and allows it to focus on more detailed segmentation of the organs and vessels. We utilize training and validation sets consisting of 331 clinical CT images and test our models on a completely unseen data collection acquired at a different hospital that includes 150 CT scans, targeting three anatomical organs (liver, spleen, and pancreas). In challenging organs such as the pancreas, our cascaded approach improves the mean Dice score from 68.5 to 82.2%, achieving the highest reported average score on this dataset. We compare with a 2D FCN method on a separate dataset of 240 CT scans with 18 classes and achieve a significantly higher performance in small organs and vessels. Furthermore, we explore fine-tuning our models to different datasets. Our experiments illustrate the promise and robustness of current 3D FCN based semantic segmentation of medical images, achieving state-of-the-art results.<sup>1</sup>.

engineering, biomedical,radiology, nuclear medicine & medical imaging
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

F. Milletarì,Seyed-Ahmad Ahmadi,N. Navab

DOI: https://doi.org/10.1109/3DV.2016.79

2016-06-15

Abstract:Convolutional Neural Networks (CNNs) have been recently employed to solve problems from both the computer vision and medical image analysis fields. Despite their popularity, most approaches are only able to process 2D images while most medical data used in clinical practice consists of 3D volumes. In this work we propose an approach to 3D image segmentation based on a volumetric, fully convolutional, neural network. Our CNN is trained end-to-end on MRI volumes depicting prostate, and learns to predict segmentation for the whole volume at once. We introduce a novel objective function, that we optimise during training, based on Dice coefficient. In this way we can deal with situations where there is a strong imbalance between the number of foreground and background voxels. To cope with the limited number of annotated volumes available for training, we augment the data applying random non-linear transformations and histogram matching. We show in our experimental evaluation that our approach achieves good performances on challenging test data while requiring only a fraction of the processing time needed by other previous methods.

Medicine,Computer Science
Scale-Equivariant Deep Learning for 3D Data

Thomas Wimmer,Vladimir Golkov,Hoai Nam Dang,Moritz Zaiss,Andreas Maier,Daniel Cremers

DOI: https://doi.org/10.48550/arXiv.2304.05864

2023-04-12

Abstract:The ability of convolutional neural networks (CNNs) to recognize objects regardless of their position in the image is due to the translation-equivariance of the convolutional operation. Group-equivariant CNNs transfer this equivariance to other transformations of the input. Dealing appropriately with objects and object parts of different scale is challenging, and scale can vary for multiple reasons such as the underlying object size or the resolution of the imaging modality. In this paper, we propose a scale-equivariant convolutional network layer for three-dimensional data that guarantees scale-equivariance in 3D CNNs. Scale-equivariance lifts the burden of having to learn each possible scale separately, allowing the neural network to focus on higher-level learning goals, which leads to better results and better data-efficiency. We provide an overview of the theoretical foundations and scientific work on scale-equivariant neural networks in the two-dimensional domain. We then transfer the concepts from 2D to the three-dimensional space and create a scale-equivariant convolutional layer for 3D data. Using the proposed scale-equivariant layer, we create a scale-equivariant U-Net for medical image segmentation and compare it with a non-scale-equivariant baseline method. Our experiments demonstrate the effectiveness of the proposed method in achieving scale-equivariance for 3D medical image analysis. We publish our code at <a class="link-external link-https" href="https://github.com/wimmerth/scale-equivariant-3d-convnet" rel="external noopener nofollow">this https URL</a> for further research and application.

Computer Vision and Pattern Recognition,Machine Learning
Visualizing MRI Deep Learning Segmentation Algorithms using 3D Printing

Greg Tyler,Oliver Mathias,Andrew Papilion

DOI: https://doi.org/10.35745/ijcmb2021v01.01.0005

2021-12-30

International Journal of Clinical Medicine and Bioengineering

Abstract:As the capabilities and roles of Artificial Intelligence (AI) in the medical field are continually expanded, new potential uses, and combinations of technology become viable. This paper highlights a methodology for utilizing AI Magnetic Resonance Imaging (MRI) segmentation networks and 3D printing processes in conjunction for medical diagnosis, planning, and visualization of medical images. We also include promising benefits and potential medical offerings made possible by this system. By training a "U- Net" on the 2019 BraTS dataset, we base our research on an MRI brain lesion segmentation dataset with sustaining performance and world recognition. This network automatically segments novel MRI scans into lesion and non-lesion regions. We pair this network with a 3D printing process that enables us to print fully segmented, 1:1 scale, patient organs aided by AI techniques to better explain cases, test models, and plan for operations. In addition to a clearly outlined process that enables these offerings, we establish a potential trajectory for how these combined tools will continue to revolutionize the ways healthcare professionals interact with patients and their data.

Long-term results of stenting of the aortic bifurcation.

3D Deep Learning on Medical Images: A Review

Super Images -- A New 2D Perspective on 3D Medical Imaging Analysis

Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data

Cross-dimensional transfer learning in medical image segmentation with deep learning

2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data

3-D Convolutional Neural Networks for Glioblastoma Segmentation

Performance of a Deep Neural Network Algorithm Based on a Small Medical Image Dataset: Incremental Impact of 3D-to-2D Reformation Combined with Novel Data Augmentation, Photometric Conversion, or Transfer Learning

One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation

A low resource 3D U-Net based deep learning model for medical image analysis

CNN-based Segmentation of Medical Imaging Data

3D Self-Supervised Methods for Medical Imaging

Pretrained Deep 2.5D Models for Efficient Predictive Modeling from Retinal OCT

Leveraging Convolutional Neural Networks for 3D Quantitative Angiography Reconstructions from Sparse Cone Beam CT Projections Utilizing CFD Data

Transferring Models Trained on Natural Images to 3D MRI via Position Encoded Slice Models

Spatiotemporal Modeling Encounters 3D Medical Image Analysis: Slice-Shift UNet with Multi-View Fusion

Comparative Evaluation of 3D and 2D Deep Learning Techniques for Semantic Segmentation in CT Scans

An application of cascaded 3D fully convolutional networks for medical image segmentation

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Scale-Equivariant Deep Learning for 3D Data

Visualizing MRI Deep Learning Segmentation Algorithms using 3D Printing