Abstract:Objective. Current segmentation practice for thoracic cancer RT considers the whole heart as a single organ despite increased risks of cardiac toxicities from irradiation of specific cardiac substructures. Segmenting up to 15 different cardiac substructures can be a very time-intensive process, especially due to their different volume sizes and anatomical variations amongst different patients. In this work, a new deep learning (DL)-based mutual enhancing strategy is introduced for accurate and automatic segmentation, especially of smaller substructures such as coronary arteries. Approach. Our proposed method consists of three subnetworks: retina U-net, classification module, and segmentation module. Retina U-net is used as a backbone network architecture that aims to learn deep features from the whole heart. Whole heart feature maps from retina U-net are then transferred to four different sets of classification modules to generate classification localization maps of coronary arteries, great vessels, chambers of the heart, and valves of the heart. Each classification module is in sync with its corresponding subsequent segmentation module in a bootstrapping manner, allowing them to share their encoding paths to generate a mutual enhancing strategy. We evaluated our method on three different datasets: institutional CT datasets (55 subjects) 2) publicly available Multi-Modality Whole Heart Segmentation (MM-WHS) challenge datasets (120 subjects), and Automated Cardiac Diagnosis Challenge (ACDC) datasets (100 subjects). For institutional datasets, we performed five-fold cross-validation on training data (45 subjects) and performed inference on separate hold-out data (10 subjects). For each subject, 15 cardiac substructures were manually contoured by a resident physician and evaluated by an attending radiation oncologist. For the MM-WHS dataset, we trained the network on 100 datasets and performed an inference on a separate hold-out dataset with 20 subjects, each with 7 cardiac substructures. For ACDC datasets, we performed five-fold cross-validation on 100 datasets, each with 3 cardiac substructures. We compared the proposed method against four different network architectures: 3D U-net, mask R-CNN, mask scoring R-CNN, and proposed network without classification module. Segmentation accuracies were statistically compared through dice similarity coefficient, Jaccard, 95% Hausdorff distance, mean surface distance, root mean square distance, center of mass distance, and volume difference. Main results. The proposed method generated cardiac substructure segmentations with significantly higher accuracy (P < 0.05) for small substructures, especially for coronary arteries such as left anterior descending artery (CA-LADA) and right coronary artery (CA-RCA) in comparison to four competing methods. For large substructures (i.e. chambers of the heart), our method yielded comparable results to mask scoring R-CNN method, resulting in significantly (P < 0.05) improved segmentation accuracy in comparison to 3D U-net and mask R-CNN. Significance. A new DL-based mutual enhancing strategy was introduced for automatic segmentation of cardiac substructures. Overall results of this work demonstrate the ability of the proposed method to improve segmentation accuracies of smaller substructures such as coronary arteries without largely compromising the segmentation accuracies of larger substructures. Fast and accurate segmentations of up to 15 substructures can possibly be used as a tool to rapidly generate substructure segmentations followed by physicians' reviews to improve clinical workflow.

Mutual enhancing learning-based automatic segmentation of CT cardiac substructure

Segmentation of Cardiac Tissues and Organs for CCTA Images Based on a Deep Learning Model

Automatic Delineation of Cardiac Substructures Using A Region-Based Fully Convolutional Network

Anatomical Prior-Based Automatic Segmentation for Cardiac Substructures from Computed Tomography Images.

Multi-Planar Deep Segmentation Networks for Cardiac Substructures from MRI and CT

Deep learning enables the fully automated quantification of high-resolution cardiac CT images: human in loop for model development

Cascaded Mutual Enhancing Networks for Brain Tumor Subregion Segmentation in Multiparametric MRI

Automated CT Segmentation for Rapid Assessment of Anatomical Variations in Head-and-neck Radiation Therapy.

Whole Heart and Great Vessel Segmentation in Congenital Heart Disease Using Deep Neural Networks and Graph Matching

Multi-class probabilistic atlas-based whole heart segmentation method in cardiac CT and MRI

An Improved Combination of Faster R-CNN and U-Net Network for Accurate Multi-Modality Whole Heart Segmentation

Graph matching and deep neural networks based whole heart and great vessel segmentation in congenital heart disease

Fully automatic cardiac four chamber and great vessel segmentation on CT pulmonary angiography using deep learning

MSMHSA-DeepLab V3+: An Effective Multi-Scale, Multi-Head Self-Attention Network for Dual-Modality Cardiac Medical Image Segmentation

MCHNet: an Efficient Cross Attention-guided Hierarchical Multi-scale Network for Segmentation of Organs at Risk in CT Images

Radiologically based automated segmentation of cardiac MRI using an improved U-Net neural algorithm

Deep anatomy learning for lung airway and artery-vein modeling with contrast-enhanced CT synthesis

Automatic Segmentation of Head-Neck Organs by Multi-mode CNNs for Radiation Therapy

CBCT Lung Multi-Oar Segmentation Via Hierarchical Network.

Synthetic CT-aided Online CBCT Multi-Organ Segmentation for CBCT-guided Adaptive Radiotherapy of Pancreatic Cancer

Deep Learning from Dual-Energy Information for Whole-Heart Segmentation in Dual-Energy and Single-Energy Non-Contrast-Enhanced Cardiac CT