Abstract:PurposeAs affordable equipment, electronic portal imaging devices (EPIDs) are wildly used in radiation therapy departments to verify patients’ positions for accurate radiotherapy. However, these devices tend to produce visually ambiguous and low‐contrast planar digital radiographs under megavoltage x ray (MV‐DRs), which poses a tremendous challenge for clinicians to perform multimodal registration between the MV‐DRs and the kilovoltage digital reconstructed radiographs (KV‐DRRs) developed from the planning computed tomography. Furthermore, the existent of strong appearance variations also makes accurate registration beyond the reach of current automatic algorithms.MethodsWe propose a novel modality conversion approach to this task that first synthesizes KV images from MV‐DRs, and then registers the synthesized and real KV‐DRRs. We focus on the synthesis technique and develop a conditional generative adversarial network with information bottleneck extension (IB‐cGAN) that takes MV‐DRs and nonaligned KV‐DRRs as inputs and outputs synthesized KV images. IB‐cGAN is designed to address two main challenges in deep‐learning‐based synthesis: (a) training with a roughly aligned dataset suffering from noisy correspondences; (b) making synthesized images have real clinical meanings that faithfully reflects MV‐DRs rather than nonaligned KV‐DRRs. Accordingly, IB‐cGAN employs (a) an adversarial loss to provide training supervision at semantic level rather than the imprecise pixel level; (b) an IB to constrain the information from the nonaligned KV‐DRRs.ResultsWe collected 2698 patient scans to train the model and 208 scans to test its performance. The qualitative results demonstrate realistic KV images can be synthesized allowing clinicians to perform the visual registration. The quantitative results show it significantly outperforms current nonmodality conversion methods by 22.37% (P = 0.0401) in terms of registration accuracy.ConclusionsThe modality conversion approach facilitates the downstream MV–KV registration for both clinicians and off‐the‐shelf registration algorithms. With this approach, it is possible to benefit the developing countries where inexpensive EPIDs are widely used for the image‐guided radiation therapy.

Multimodal Image Registration Based on Binary Gradient Angle Descriptor.

IMAGE REGISTRATION BASED ON MAXIMIZATION OF GRADIENT CODE MUTUAL INFORMATION

Multi-Modality Image Registration with Gradient Orientation Information Based on Entropic Spanning Graph

Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Fast and Robust Multimodal Image Registration Using A Local Derivative Pattern

Multimodal Medical Image Registration Based on Gradient Vector Flow and Particle Swarm Optimization

Edge Preserving Multi-Modal Registration Based On Gradient Intensity Self-Similarity

Boosting Structure Consistency for Multispectral and Multimodal Image Registration

Unsupervised MMRegNet Based on Spatially Encoded Gradient Information.

multiGradICON: A Foundation Model for Multimodal Medical Image Registration

Multi-Modality Medical Image Registration Based on Regional Joint Rényi Entropy

Research of Non-rigid Medical Image Registration Based on Gradient Information

A Modality Conversion Approach to MV-DRs and KV-DRRs Registration Using Information Bottlenecked Conditional Generative Adversarial Network

An Image Registration Method Based on the Combination NGVF Information and Mutual Information

Normalized Total Gradient: A New Measure for Multispectral Image Registration

Influence of Implementation Parameters on Registration of Mr and Spect Brain Images by Maximization of Mutual Information

Multimodality image registration by maximization of mutual information

Multiscale Approach for Variational Problem Joint Diffeomorphic Image Registration and Intensity Correction: Theory and Application

Multimodal image registration techniques: a comprehensive survey

Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study