Abstract:Complicated image registration is a key issue in medical image analysis, and deep learning-based methods have achieved better results than traditional methods. The methods include ConvNet-based and Transformer-based methods. Although ConvNets can effectively utilize local information to reduce redundancy via small neighborhood convolution, the limited receptive field results in the inability to capture global dependencies. Transformers can establish long-distance dependencies via a self-attention mechanism; however, the intense calculation of the relationships among all tokens leads to high redundancy. We propose a novel unsupervised image registration method named the unified Transformer and superresolution (UTSRMorph) network, which can enhance feature representation learning in the encoder and generate detailed displacement fields in the decoder to overcome these problems. We first propose a fusion attention block to integrate the advantages of ConvNets and Transformers, which inserts a ConvNet-based channel attention module into a multihead self-attention module. The overlapping attention block, a novel cross-attention method, uses overlapping windows to obtain abundant correlations with match information of a pair of images. Then, the blocks are flexibly stacked into a new powerful encoder. The decoder generation process of a high-resolution deformation displacement field from low-resolution features is considered as a superresolution process. Specifically, the superresolution module was employed to replace interpolation upsampling, which can overcome feature degradation. UTSRMorph was compared to state-of-the-art registration methods in the 3D brain MR (OASIS, IXI) and MR-CT datasets. The qualitative and quantitative results indicate that UTSRMorph achieves relatively better performance. The code and datasets are publicly available at <a class="link-external link-https" href="https://github.com/Runshi-Zhang/UTSRMorph" rel="external noopener nofollow">this https URL</a>.

SuperCUT, an unsupervised multimodal image registration with deep learning for biomedical microscopy

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Unsupervised deep learning registration model for multimodal brain images

End-to-end multimodal image registration via reinforcement learning

Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance

Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization

Weakly-supervised convolutional neural networks for multimodal image registration

Explainable unsupervised multi-modal image registration using deep networks

Unsupervised deep learning for geometric feature detection and multilevel-multimodal image registration

Unsupervised Histological Image Registration Using Structural Feature Guided Convolutional Neural Network.

An Unsupervised Convolution Neural Network for Deformable Registration of Mono/Multi-Modality Medical Images

UTSRMorph: A Unified Transformer and Superresolution Network for Unsupervised Medical Image Registration

3D Biological/Biomedical Image Registration with enhanced Feature Extraction and Outlier Detection

Unsupervised Non-rigid Histological Image Registration Guided by Keypoint Correspondences Based on Learnable Deep Features with Iterative Training

Mutual information neural estimation for unsupervised multi-modal registration of brain images

Framework for Deep Learning Based Multi-Modality Image Registration of Snapshot and Pathology Images

Matching in the Wild: Learning Anatomical Embeddings for Multi-Modality Images

Unsupervised Deformable Image Registration Using Cycle-Consistent CNN

Deep learning based registration of serial whole-slide histopathology images in different stains

Unsupervised deep learning enables real-time image registration of fast-scanning optical-resolution photoacoustic microscopy

A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention