Abstract:Objective: Dynamic cone-beam computed tomography (CBCT) can capture high-spatial-resolution, time-varying images for motion monitoring, patient setup, and adaptive planning of radiotherapy. However, dynamic CBCT reconstruction is an extremely ill-posed spatiotemporal inverse problem, as each CBCT volume in the dynamic sequence is only captured by one or a few X-ray projections, due to the slow gantry rotation speed and the fast anatomical motion (e.g., breathing). Approach: We developed a machine learning-based technique, prior-model-free spatiotemporal implicit neural representation (PMF-STINR), to reconstruct dynamic CBCTs from sequentially acquired X-ray projections. PMF-STINR employs a joint image reconstruction and registration approach to address the under-sampling challenge, enabling dynamic CBCT reconstruction from singular X-ray projections. Specifically, PMF-STINR uses spatial implicit neural representations to reconstruct a reference CBCT volume, and it applies temporal INR to represent the intra-scan dynamic motion with respect to the reference CBCT to yield dynamic CBCTs. PMF-STINR couples the temporal INR with a learning-based B-spline motion model to capture time-varying deformable motion during the reconstruction. Compared with the previous methods, the spatial INR, the temporal INR, and the B-spline model of PMF-STINR are all learned on the fly during reconstruction in a one-shot fashion, without using any patient-specific prior knowledge or motion sorting/binning. Main results: PMF-STINR was evaluated via digital phantom simulations, physical phantom measurements, and a multi-institutional patient dataset featuring various imaging protocols (half-fan/full-fan, full sampling/sparse sampling, different energy and mAs settings, etc.). The results showed that the one-shot learning-based PMF-STINR can accurately and robustly reconstruct dynamic CBCTs and capture highly irregular motion with high temporal (~0.1s) resolution and sub-millimeter accuracy. Significance: PMF-STINR can reconstruct dynamic CBCTs and solve the intra-scan motion problem from conventional 3D CBCT scans without using any prior anatomical/motion model or motion sorting/binning. It can be a promising tool for motion management by offering richer motion information than traditional 4D-CBCTs.

Neural space-time model for dynamic scene recovery in multi-shot computational imaging systems

Neural space–time model for dynamic multi-shot imaging

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Neural Computed Tomography

Neural rendering enables dynamic tomography

Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera

A Memory-Efficient Dynamic Image Reconstruction Method using Neural Fields

Deep Reconstruction Model For Dynamic Pet Images

Enhancing Dynamic CT Image Reconstruction with Neural Fields and Optical Flow

Dynamic Reconstruction from Neuromorphic Data

Non-line-of-Sight Imaging via Neural Transient Fields

Dynamic CBCT imaging using prior model-free spatiotemporal implicit neural representation (PMF-STINR)

Spatial-temporal Nonparametric Background Subtraction in Dynamic Scenes

Dynamic SPECT reconstruction with temporal edge correlation

Real-time 3D Nanoscale Coherent Imaging via Physics-aware Deep Learning

Distributed Stochastic Optimization of a Neural Representation Network for Time-Space Tomography Reconstruction

Self-Supervised Mutual Learning for Dynamic Scene Reconstruction of Spiking Camera.

Plug-and-Play Algorithms for Dynamic Non-line-of-sight Imaging

Capture the Moment: High-Speed Imaging With Spiking Cameras Through Short-Term Plasticity

DRSM: efficient neural 4d decomposition for dynamic reconstruction in stationary monocular cameras