Abstract:Background: Current commercially available hybrid magnetic resonance linear accelerators (MR-Linac) use 2D+t cine MR imaging to provide intra-fractional motion monitoring. However, given the limited temporal resolution of cine MR imaging, target intra-frame motion deterioration effects, resulting in effective time latency and motion artifacts in the image domain, can be appreciable, especially in the case of fast breathing. Purpose: The aim of this work is to investigate intra-frame motion deterioration effects in MR-guided radiotherapy (MRgRT) by simulating the motion-corrupted image acquisition, and to explore the feasibility of deep-learning-based compensation approaches, relying on the intra-frame motion information which is spatially and temporally encoded in the raw data (k-space). Methods: An intra-frame motion model was defined to simulate motion-corrupted MR images, with 4D anthropomorphic digital phantoms being exploited to provide ground truth 2D+t cine MR sequences. A total number of 10 digital phantoms were generated for lung cancer patients, with randomly selected eight patients for training or validation and the remaining two for testing. The simulation code served as the data generator, and a dedicated motion pattern perturbation scheme was proposed to build the intra-frame motion database, where three degrees of freedom were designed to guarantee the diversity of intra-frame motion trajectories, enabling a thorough exploration in the domain of the potential anatomical structure positions. U-Nets with three types of loss functions: L1 or L2 loss defined in image or Fourier domain, referred to as NNImgLoss-L1 , NNFloss-L1 and NNL2-Loss were trained to extract information from the motion-corrupted image and used to estimate the ground truth final-position image, corresponding to the end of the acquisition. Images before and after compensation were evaluated in terms of (i) image mean-squared error (MSE) and mean absolute error (MAE), and (ii) accuracy of gross tumor volume (GTV) contouring, based on optical-flow image registration. Results: Image degradation caused by intra-frame motion was observed: for a linearly and fully acquired Cartesian readout k-space trajectory, intra-frame motion resulted in an imaging latency of approximately 50% of the acquisition time; in comparison, the motion artifacts exhibited only a negligible contribution to the overall geometric errors. All three compensation models led to a decrease in image MSE/MAE and GTV position offset compared to the motion-corrupted image. In the investigated testing dataset for GTV contouring, the average dice similarity coefficients (DSC) improved from 88% to 96%, and the 95th percentile Hausdorff distance (HD95 ) dropped from 4.8 mm to 2.1 mm. Different models showed slight performance variations across different intra-frame motion amplitude categories: NNImgLoss-L1 excelled for small/medium amplitudes, whereas NNFloss-L1 demonstrated higher DSC median values at larger amplitudes. The saliency maps of the motion-corrupted image highlighted the major contribution of the later acquired k-space data, as well as the edges of the moving anatomical structures at their final positions, during the model inference stage. Conclusions: Our results demonstrate the deep-learning-based approaches have the potential to compensate for intra-frame motion by utilizing the later acquired data to drive the convergence of the earlier acquired k-space components.

Future frame prediction in chest cine MR imaging using the PCA respiratory motion model and dynamically trained recurrent neural networks

Prediction of the motion of chest internal points using a recurrent neural network trained with real-time recurrent learning for latency compensation in lung cancer radiotherapy

Respiratory motion forecasting with online learning of recurrent neural networks for safety enhancement in externally guided radiotherapy

Predicting respiratory motion using a novel patient specific dual deep recurrent neural networks

Intra-frame motion deterioration effects and deep-learning-based compensation in MR-guided radiotherapy

Online Prediction for Respiratory Movement Compensation: a Patient-Specific Gating Control for MRI-guided Radiotherapy.

Imaging error reduction in radial cine-MRI with deep learning-based intra-frame motion compensation

Prediction of the position of external markers using a recurrent neural network trained with unbiased online recurrent optimization for safe lung cancer radiotherapy

Predicting Real-Time 3D Deformation Field Maps (DFM) Based on Volumetric Cine MRI (VC-MRI) and Artificial Neural Networks for On-Board 4D Target Tracking: a Feasibility Study

Fast and Accurate Online Sequential Learning of Respiratory Motion with Random Convolution Nodes for Radiotherapy Applications.

Improving respiratory signal prediction with a deep neural network and simple changes to the input and output data format

Real-Time 2D MR Cine from Beam Eye’s View with Tumor-Volume Projection to Ensure Beam-to-Tumor Conformality for MR-Guided Radiotherapy of Lung Cancer

Volumetric prediction of breathing and slow drifting motion in the abdomen using radial MRI and multi-temporal resolution modeling

A Real-Time Four-Dimensional Reconstruction Algorithm of Cine-Magnetic Resonance Imaging (Cine-MRI) Using Deep Learning

Dynamic recurrent inference machines for accelerated MRI-guided radiotherapy of the liver

Respiratory Motion Prediction with Empirical Mode Decomposition-Based Random Vector Functional Link

Towards real-time respiratory motion prediction based on long short-term memory neural networks

Predicting 4D Liver MRI for MR-guided Interventions

Motion-flow-guided recurrent network for respiratory signal estimation of x-ray angiographic image sequences

Prediction of in-plane organ deformation during free-breathing radiotherapy via discriminative spatial transformer networks

Prediction of Lung Motion from Four-Dimensional Computer Tomography (4DCT) Images Using Bayesian Registration and Trajectory Modelling