Abstract:When conducting spine-related diagnosis and surgery, the three-dimensional (3D) upright posture of the spine under natural weight bearing is of significant clinical value for physicians to analyze the force on the spine. However, existing medical imaging technologies cannot meet current requirements of medical service. On the one hand, the mainstream 3D volumetric imaging modalities (e.g. CT and MRI) require patients to lie down during the imaging process. On the other hand, the imaging modalities conducted in an upright posture (e.g. radiograph) can only realize 2D projections, which lose the valid information of spinal anatomy and curvature. Developments of deep learning-based 3D reconstruction methods bring potential to overcome the limitations of the existing medical imaging technologies. To deal with the limitations of current medical imaging technologies as is described above, in this paper, we propose a novel deep learning framework, ReVerteR , which can realize automatic 3D Re construction of Verte brae from orthogonal bi-planar R adiographs. With the utilization of self-attention mechanism and specially designed loss function combining Dice, Hausdorff, Focal, and MSE, ReVerteR can alleviate the sample-imbalance problem during the reconstruction process and realize the fusion of the centroid annotation and the focused vertebra. Furthermore, aiming at automatic and customized 3D spinal reconstruction in real-world scenarios, we extend ReVerteR to a clinical deployment-oriented framework, and develop an interactive interface with all functions in the framework integrated so as to enhance human–computer interaction during clinical decision-making. Extensive experiments and visualization conducted on our constructed datasets based on two benchmark datasets of spinal CT, VerSe 2019 and VerSe 2020, demonstrate the effectiveness of our proposed ReVerteR. In this paper, we propose an automatic 3D reconstruction method of vertebrae based on orthogonal bi-planar radiographs. With the 3D upright posture of the spine under natural weight bearing effectively constructed, our proposed method is expected to better support doctors make clinical decision during spine-related diagnosis and surgery.

3D shape reconstruction of lumbar vertebra from two X-ray images and a CT model

An Accurately Represented Finite Element Model of Lumbar Motion Segment

Reconstruction of 3D Lumvar Vertebra from Two X-ray Images Based on 2D/3D Registration

2D/3D Reconstruction of a Scaled Lumbar Vertebral Model from a Single Fluoroscopic Image.

Reconstruct Spine CT from Biplanar X-Rays via Diffusion Learning

Scaled, patient-specific 3D vertebral model reconstruction based on 2D lateral fluoroscopy

Three-Dimensional Vertebrae Reconstruction from Stereoradiography Based on Epipolar Geometry

A Simple Approach for 3D Reconstruction of the Spine from Biplanar Radiography.

X23D—Intraoperative 3D Lumbar Spine Shape Reconstruction Based on Sparse Multi-View X-ray Data

2D/3D Shape Model Registration with X-ray Images for Patient-Specific Spine Geometry Reconstruction.

Statistical Shape and Deformation Models Based 2D–3D Reconstruction

Anatomical prior based vertebra modelling for reappearance of human spines

3-D Reconstruction of the Spine from Biplanar Radiographs Based on Contour Matching Using the Hough Transform

Automatic 3D reconstruction of vertebrae from orthogonal bi-planar radiographs

3 D reconstruction of femur based on 2 D/3 D registration

3D human mandible reconstruction from CT data

Reconstruction of 3D lumbar spine models from incomplete segmentations using landmark detection

Assessment and Improvement of a Novel ultrasound-based 3D Reconstruction Method: Registered for Lumbar Spine

Data-driven Method for 3D Axis-symmetric Object Reconstruction from Single Cone-beam Projection Data

3D Spine Model Reconstruction Based on RGBD Images of Unclothed Back Surface

3D Spine Shape Estimation from Single 2D DXA