Abstract:Dynamic three-dimensional (4D) reconstruction from two-dimensional X-ray coronary angiography (CA) remains a significant clinical problem. Challenges include sparse-view settings, intra-scan motion, and complex vessel morphology such as structure sparsity and background occlusion. Existing CA reconstruction methods often require extensive user interaction or large training datasets. On the other hand, Neural Radiance Field (NeRF), a promising deep learning technique, has successfully reconstructed high-fidelity static scenes for natural and medical scenes. Recent work, however, identified that sparse-views, background occlusion, and dynamics still pose a challenge when applying NeRF in the X-ray angiography context. Meanwhile, many successful works for natural scenes propose regularization for sparse-view reconstruction or scene decomposition to handle dynamics. However, these techniques do not directly translate to the CA context, where both challenges and background occlusion are significant. This paper introduces NeRF-CA, the first step toward a 4D CA reconstruction method that achieves reconstructions from sparse coronary angiograms with cardiac motion. We leverage the motion of the coronary artery to decouple the scene into a dynamic coronary artery component and static background. We combine this scene decomposition with tailored regularization techniques. These techniques enforce the separation of the coronary artery from the background by enforcing dynamic structure sparsity and scene smoothness. By uniquely combining these approaches, we achieve 4D reconstructions from as few as four angiogram sequences. This setting aligns with clinical workflows while outperforming state-of-the-art X-ray sparse-view NeRF reconstruction techniques. We validate our approach quantitatively and qualitatively using 4D phantom datasets and ablation studies.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is **reconstructing the dynamic three - dimensional (4D) structure from two - dimensional X - ray coronary angiography (CA) images with extremely sparse views**. Specifically, the paper faces the following challenges: 1. **Sparse - view setting**: During the standard CA interventional procedure, only a limited number of angiographic image views are obtained for each patient, which is known as sparse views. Such sparse views lead to the problem of insufficient information during the reconstruction process. 2. **Intra - scan motion**: Due to cardiac and respiratory movements, the coronary arteries will move during the scan, which brings additional complexity to the reconstruction. 3. **Complex vascular morphology**: The coronary artery structure is sparse and the background may be occluded, and these factors further increase the difficulty of reconstruction. Existing CA reconstruction methods usually require a large amount of user interaction or a large amount of training data, which is impractical in clinical practice. Although Neural Radiance Field (NeRF) has achieved success in natural and medical scenarios, its application in X - ray coronary angiography still faces the above challenges. To address these problems, the paper proposes the NeRF - CA method, which achieves 4D reconstruction from extremely sparse coronary angiography sequences through the following techniques: 1. **Scene decomposition**: Utilize the rapid movement of the coronary arteries to decompose the scene into dynamic coronary artery components and static background components. 2. **Regularization techniques**: Combine techniques such as weighted pixel sampling, static and dynamic factorization regularization, and entropy regularization to ensure accurate separation of the dynamic coronary arteries from the background. 3. **Sparse - view reconstruction**: Use coarse - to - fine position encoding and occlusion regularization techniques to improve the reconstruction effect under sparse views. Through the combination of these techniques, NeRF - CA can achieve high - quality 4D reconstruction from a minimum of four coronary angiography sequences, conform to the clinical workflow, and is significantly superior to existing X - ray sparse - view NeRF reconstruction methods.

NeRF-CA: Dynamic Reconstruction of X-ray Coronary Angiography with Extremely Sparse-views

Structure-Aware Sparse-View X-ray 3D Reconstruction

$ρ$-NeRF: Leveraging Attenuation Priors in Neural Radiance Field for 3D Computed Tomography Reconstruction

NeRF-US: Removing Ultrasound Imaging Artifacts from Neural Radiance Fields in the Wild

TomoGRAF: A Robust and Generalizable Reconstruction Network for Single-View Computed Tomography

UC-NeRF: Uncertainty-aware Conditional Neural Radiance Fields from Endoscopic Sparse Views

SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimates

Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations

ACnerf: enhancement of neural radiance field by alignment and correction of pose to reconstruct new views from a single x-ray

CAD-NeRF: Learning NeRFs from Uncalibrated Few-view Images by CAD Model Retrieval

ColonNeRF: High-Fidelity Neural Reconstruction of Long Colonoscopy

CTNeRF: Cross-Time Transformer for Dynamic Neural Radiance Field from Monocular Video

NeRF-OR: neural radiance fields for operating room scene reconstruction from sparse-view RGB-D videos

BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields

NeRF: Neural Radiance Field in 3D Vision, A Comprehensive Review

Dense Depth Priors for Neural Radiance Fields from Sparse Input Views

3DGR-CAR: Coronary artery reconstruction from ultra-sparse 2D X-ray views with a 3D Gaussians representation

ProxNF: Neural Field Proximal Training for High-Resolution 4D Dynamic Image Reconstruction

CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory

NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction

Enhancing Neural Radiance Fields with Depth and Normal Completion Priors from Sparse Views