Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

Zhe Xu,Jie Luo,Jiangpeng Yan,Ritvik Pulya,Xiu Li,William Wells III,Jayender Jagadeesan
DOI: https://doi.org/10.48550/arXiv.2007.02790
2020-09-21
Abstract:Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?