Abstract:Panoramic X-ray (PX) is a prevalent modality in dentistry practice owing to its wide availability and low cost. However, as a 2D projection of a 3D structure, PX suffers from anatomical information loss and PX diagnosis is limited compared to that with 3D imaging modalities. 2D-to-3D reconstruction methods have been explored for the ability to synthesize the absent 3D anatomical information from 2D PX for use in PX image analysis. However, there are challenges in leveraging such 3D synthesized reconstructions. First, inferring 3D depth from 2D images remains a challenging task with limited accuracy. The second challenge is the joint analysis of 2D PX with its 3D synthesized counterpart, with the aim to maximize the 2D-3D synergy while minimizing the errors arising from the synthesized image. In this study, we propose a new method termed 3DPX - PX image analysis guided by 2D-to-3D reconstruction, to overcome these challenges. 3DPX consists of (i) a novel progressive reconstruction network to improve 2D-to-3D reconstruction and, (ii) a contrastive-guided bidirectional multimodality alignment module for 3D-guided 2D PX classification and segmentation tasks. The reconstruction network progressively reconstructs 3D images with knowledge imposed on the intermediate reconstructions at multiple pyramid levels and incorporates Multilayer Perceptrons to improve semantic understanding. The downstream networks leverage the reconstructed images as 3D anatomical guidance to the PX analysis through feature alignment, which increases the 2D-3D synergy with bidirectional feature projection and decease the impact of potential errors with contrastive guidance. Extensive experiments on two oral datasets involving 464 studies demonstrate that 3DPX outperforms the state-of-the-art methods in various tasks including 2D-to-3D reconstruction, PX classification and lesion segmentation.

Draw Sketch, Draw Flesh: Whole-Body Computed Tomography from Any X-Ray Views

Deep Fusion Network Based Sparse View CT Reconstructions for Clinical Diagnostic Scanners

2D-to-3D: A Review for Computational 3D Image Reconstruction from X-ray Images

XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors

3D Reconstruction of 2D X-Ray Images

Body Composition Assessment with Limited Field-of-view Computed Tomography: A Semantic Image Extension Perspective

3DDX: Bone Surface Reconstruction from a Single Standard-Geometry Radiograph via Dual-Face Depth Estimation

X-ray Dissectography Enables Stereotography to Improve Diagnostic Performance

MSKdeX: Musculoskeletal (MSK) decomposition from an X-ray image for fine-grained estimation of lean muscle mass and muscle volume

MaSkel: A Model for Human Whole-body X-rays Generation from Human Masking Images

Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans

Recognition and 3D Visualization of Human Body Parts and Bone Areas Using CT Images

Oral-3D: Reconstructing the 3D Bone Structure of Oral Cavity from 2D Panoramic X-ray

Method for reconstructing three-dimensional body image through two-dimensional X ray image and application thereof

Reconstruct Spine CT from Biplanar X-Rays via Diffusion Learning

DIFR3CT: Latent Diffusion for Probabilistic 3D CT Reconstruction from Few Planar X-Rays

DX2CT: Diffusion Model for 3D CT Reconstruction from Bi or Mono-planar 2D X-ray(s)

3DPX: Single Panoramic X-ray Analysis Guided by 3D Oral Structure Reconstruction

TomoRay: Generating synthetic computed tomography of the spine from biplanar radiographs

Full-length-body CBCT imaging in upright position with robotic-arm system: a simulation study

Spatial-Division Augmented Occupancy Field for Bone Shape Reconstruction from Biplanar X-Rays