Abstract:PurposeAs affordable equipment, electronic portal imaging devices (EPIDs) are wildly used in radiation therapy departments to verify patients’ positions for accurate radiotherapy. However, these devices tend to produce visually ambiguous and low‐contrast planar digital radiographs under megavoltage x ray (MV‐DRs), which poses a tremendous challenge for clinicians to perform multimodal registration between the MV‐DRs and the kilovoltage digital reconstructed radiographs (KV‐DRRs) developed from the planning computed tomography. Furthermore, the existent of strong appearance variations also makes accurate registration beyond the reach of current automatic algorithms.MethodsWe propose a novel modality conversion approach to this task that first synthesizes KV images from MV‐DRs, and then registers the synthesized and real KV‐DRRs. We focus on the synthesis technique and develop a conditional generative adversarial network with information bottleneck extension (IB‐cGAN) that takes MV‐DRs and nonaligned KV‐DRRs as inputs and outputs synthesized KV images. IB‐cGAN is designed to address two main challenges in deep‐learning‐based synthesis: (a) training with a roughly aligned dataset suffering from noisy correspondences; (b) making synthesized images have real clinical meanings that faithfully reflects MV‐DRs rather than nonaligned KV‐DRRs. Accordingly, IB‐cGAN employs (a) an adversarial loss to provide training supervision at semantic level rather than the imprecise pixel level; (b) an IB to constrain the information from the nonaligned KV‐DRRs.ResultsWe collected 2698 patient scans to train the model and 208 scans to test its performance. The qualitative results demonstrate realistic KV images can be synthesized allowing clinicians to perform the visual registration. The quantitative results show it significantly outperforms current nonmodality conversion methods by 22.37% (P = 0.0401) in terms of registration accuracy.ConclusionsThe modality conversion approach facilitates the downstream MV–KV registration for both clinicians and off‐the‐shelf registration algorithms. With this approach, it is possible to benefit the developing countries where inexpensive EPIDs are widely used for the image‐guided radiation therapy.

Fast Registration Algorithm of MVCT and KVCT Based on Mutual Information.

Registration between MVCT reconstructed from EPID and kVCT.

Study on registration algorithm for portal images and simulation images in megavolt radiotherapy

A Modality Conversion Approach to MV-DRs and KV-DRRs Registration Using Information Bottlenecked Conditional Generative Adversarial Network

A Fast Algorithm to Estimate Mutual Information for Image Registration

Registration Method for CT-MR Image Based on Mutual Information

A Novel Approach for Position Verification and Dose Calculation through Local MVCT Reconstruction

A Stationary Wavelet Transform Based Approach to Registration of Planning CT and Setup Cone beam-CT Images in Radiotherapy

Influence of Implementation Parameters on Registration of Mr and Spect Brain Images by Maximization of Mutual Information

Automatic 2D/2D Registration Using Multiresolution Pyramid Based Mutual Information in Image Guided Radiation Therapy

A Hybrid Optimization Method for Maximum Mutual Information Registration

Study on the Method of Medical Rigid Image Registration Based on Mutual Information

Visible and Infrared Image Automatic Registration Algorithm Using Mutual Information

Computation of Mutual Information in Medical Image Registration Based on Mutual Information

Multimodality image registration by maximization of mutual information

Unifying Energy Minimization And Mutual Information Maximization For Robust 2d/3d Registration Of X-Ray And Ct Images

A novel 3D volumetric voxel registration technique for volume-view-guided image registration of multiple imaging modalities.

Non-rigid Registration of Multiphase Contrast-Enhanced Abdomen CT Image

A Fast Image Guide Registration Supported by Single Direction Projected CBCT

HEAD MRI Registration Based on Mutual Information

Registration Of Brain Mri/Pet Images Based On Adaptive Combination Of Intensity And Gradient Field Mutual Information