Abstract:Spinal surgery planning necessitates automatic segmentation of vertebrae in cone-beam computed tomography (CBCT), an intraoperative imaging modality that is widely used in intervention. However, CBCT images are of low-quality and artifact-laden due to noise, poor tissue contrast, and the presence of metallic objects, causing vertebra segmentation, even manually, a demanding task. In contrast, there exists a wealth of artifact-free, high quality CT images with vertebra annotations. This motivates us to build a CBCT vertebra segmentation model using unpaired CT images with annotations. To overcome the domain and artifact gaps between CBCT and CT, it is a must to address the three heterogeneous tasks of vertebra segmentation, artifact reduction and modality translation all together. To this, we propose a novel anatomy-aware artifact disentanglement and segmentation network (A$^3$DSegNet) that intensively leverages knowledge sharing of these three tasks to promote learning. Specifically, it takes a random pair of CBCT and CT images as the input and manipulates the synthesis and segmentation via different decoding combinations from the disentangled latent layers. Then, by proposing various forms of consistency among the synthesized images and among segmented vertebrae, the learning is achieved without paired (i.e., anatomically identical) data. Finally, we stack 2D slices together and build 3D networks on top to obtain final 3D segmentation result. Extensive experiments on a large number of clinical CBCT (21,364) and CT (17,089) images show that the proposed A$^3$DSegNet performs significantly better than state-of-the-art competing methods trained independently for each task and, remarkably, it achieves an average Dice coefficient of 0.926 for unpaired 3D CBCT vertebra segmentation.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the automatic segmentation of vertebrae in cone - beam computed tomography (CBCT) images. Due to the low - quality CBCT images and the presence of artifacts, even manual segmentation of vertebrae is a challenging task. However, there are a large number of artifact - free, high - quality CT images and their vertebrae - labeled data. This inspired the authors to use unpaired CT images and labels to construct a CBCT vertebrae - segmentation model. To overcome the domain gap and the artifact gap between CBCT and CT, three heterogeneous tasks must be solved simultaneously: vertebrae segmentation, artifact reduction, and modality conversion. To this end, the authors proposed a new Anatomy - Aware Artifact Disentanglement and Segmentation Network (A3DSegNet), which promotes learning by sharing knowledge among these tasks. Specifically, this network accepts a random pair of CBCT and CT images as input, and manipulates synthesis and segmentation through different decoding combinations from the disentangled latent layers. By proposing various forms of consistency between the synthesized images and the segmented vertebrae, learning without paired (i.e., anatomically identical) data is achieved. Finally, by stacking 2D slices and building a 3D network on this basis, the final 3D segmentation results are obtained. A large number of clinical CBCT (21,364) and CT (17,089) image experiments show that the proposed A3DSegNet significantly outperforms the state - of - the - art methods trained independently for each task, and achieves a performance with an average Dice coefficient of 0.926 in unpaired 3D CBCT vertebrae segmentation.

A$^3$DSegNet: Anatomy-aware artifact disentanglement and segmentation network for unpaired segmentation, artifact reduction, and modality translation

A3DSegNet: Anatomy-Aware Artifact Disentanglement and Segmentation Network for Unpaired Segmentation, Artifact Reduction, and Modality Translation

3D Graph-S<SUP>2</SUP>Net: Shape-Aware Self-ensembling Network for Semi-supervised Segmentation with Bilateral Graph Convolution

Ca-Segresnet: A Context-Aware Segmentation Residual Network for Automatic Segmentation of Identified Vertebral Bones from Ct Images

A Robust Segmentation Method Based on Improved U-Net

A Novel Deep Learning Pipeline for Vertebra Labeling and Segmentation of Spinal Computed Tomography Images

Atrous Residual Interconnected Encoder to Attention Decoder Framework for Vertebrae Segmentation via 3D Volumetric CT Images

AFSegNet: few-shot 3D ankle-foot bone segmentation via hierarchical feature distillation and multi-scale attention and fusion

Automatic Localization And Segmentation Of Vertebral Bodies In 3d Ct Volumes With Deep Learning

EG-Trans3DUNet: A Single-Staged Transformer-Based Model for Accurate Vertebrae Segmentation from Spinal Ct Images

ECSU-Net: an Embedded Clustering Sliced U-Net Coupled with Fusing Strategy for Efficient Intervertebral Disc Segmentation and Classification

Architecture Analysis and Benchmarking of 3D U-Shaped Deep Learning Models for Thoracic Anatomical Segmentation

mm3DSNet: multi-scale and multi-feedforward self-attention 3D segmentation network for CT scans of hepatobiliary ducts

Deep Geodesic Learning for Segmentation and Anatomical Landmarking

KCB-Net: A 3D knee cartilage and bone segmentation network via sparse annotation

Enhancing Single-Slice Segmentation with 3D-to-2D Unpaired Scan Distillation

D-Net: Siamese based Network with Mutual Attention for Volume Alignment

DCA: Densely Cross-scale Attention Network for Anatomically-plausible Medical Image Segmentation.

Cervical Spondylotic Myelopathy Segmentation Using Shape-Aware U-net

Anatomy Segmentation Evaluation with Sparse Ground Truth Data.