Abstract:Objective: The goal of this work was to methodically evaluate, optimize, and validate a self-supervised machine learning algorithm capable of real-time automatic registration and fluoroscopic localization of the spine using a single radiograph or fluoroscopic frame. Methods: The authors propose a two-dimensional to three-dimensional (2D-3D) registration algorithm that maximizes an image similarity metric between radiographic images to identify the position of a C-arm relative to a 3D volume. This work utilizes digitally reconstructed radiographs (DRRs), which are synthetic radiographic images generated by simulating the x-ray projections as they would pass through a CT volume. To evaluate the algorithm, the authors used cone-beam CT data for 127 patients obtained from an open-source de-identified registry of cervical, thoracic, and lumbar scans. They systematically evaluated and tuned the algorithm, then quantified the convergence rate of the model by simulating C-arm registrations with 80 randomly simulated DRRs for each CT volume. The endpoints of this study were time to convergence, accuracy of convergence for each of the C-arm's degrees of freedom, and overall registration accuracy based on a voxel-by-voxel measurement. Results: A total of 10,160 unique radiographic images were simulated from 127 CT scans. The algorithm successfully converged to the correct solution 82% of the time with an average of 1.96 seconds of computation. The radiographic images for which the algorithm converged to the solution demonstrated 99.9% registration accuracy despite utilizing only single-precision computation for speed. The algorithm was found to be optimized for convergence when the search space was limited to a ± 45° offset in the right anterior oblique/left anterior oblique, cranial/caudal, and receiver rotation angles with the radiographic isocenter contained within 8000 cm3 of the volumetric center of the CT volume. Conclusions: The investigated machine learning algorithm has the potential to aid surgeons in level localization, surgical planning, and intraoperative navigation through a completely automated 2D-3D registration process. Future work will focus on algorithmic optimizations to improve the convergence rate and speed profile.

Anatomy Detection and Localization in 3D Medical Images

Robust Multi-scale Anatomical Landmark Detection in Incomplete 3D-CT Data

Anatomic-landmark Detection Using Graphical Context Modelling.

Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images

A model-based, semi-global segmentation approach for automatic 3-D point landmark localization in neuroimages

Automatic localization of anatomical landmarks in head cine fluoroscopy images via deep learning

3D Reconstruction and Segmentation of Dissection Photographs for MRI-free Neuropathology

ConvNet-Based Localization of Anatomical Structures in 3D Medical Images

Recognition and 3D Visualization of Human Body Parts and Bone Areas Using CT Images

Robust Segmentation of Various Anatomies in 3D Ultrasound Using Hough Forests and Learned Data Representations

Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlas

Structure-Aware Long Short-Term Memory Network for 3D Cephalometric Landmark Detection

Machine learning for automated and real-time two-dimensional to three-dimensional registration of the spine using a single radiograph

CTARR: A fast and robust method for identifying anatomical regions on CT images via atlas registration

Deep Learning Body Region Classification of MRI and CT examinations

A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning

Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology

Cephalometric Landmark Regression with Convolutional Neural Networks on 3D Computed Tomography Data

Regression Segmentation for $M^{3}$ Spinal Images

Robust Landmark Detection for Alignment of Mouse Brain Section Images

Anatomy-guided multimodal registration by learning segmentation without ground truth: Application to intraprocedural CBCT/MR liver segmentation and registration