Abstract:The heart is a relatively complex non-rigid motion organ in the human body. Quantitative motion analysis of the heart takes on a critical significance to help doctors with accurate diagnosis and treatment. Moreover, cardiovascular magnetic resonance imaging (CMRI) can be used to perform a more detailed quantitative analysis evaluation for cardiac diagnosis. Deformable image registration (DIR) has become a vital task in biomedical image analysis since tissue structures have variability in medical images. Recently, the model based on masked autoencoder (MAE) has recently been shown to be effective in computer vision tasks. Vision Transformer has the context aggregation ability to restore the semantic information in the original image regions by using a low proportion of visible image patches to predict the masked image patches. A novel Transformer-ConvNet architecture is proposed in this study based on MAE for medical image registration. The core of the Transformer is designed as a masked autoencoder (MAE) and a lightweight decoder structure, and feature extraction before the downstream registration task is transformed into the self-supervised learning task. This study also rethinks the calculation method of the multi-head self-attention mechanism in the Transformer encoder. We improve the query-key-value-based dot product attention by introducing both depthwise separable convolution (DWSC) and squeeze and excitation (SE) modules into the self-attention module to reduce the amount of parameter computation to highlight image details and maintain high spatial resolution image features. In addition, concurrent spatial and channel squeeze and excitation (scSE) module is embedded into the CNN structure, which also proves to be effective for extracting robust feature representations. The proposed method, called MAE-TransRNet, has better generalization. The proposed model is evaluated on the cardiac short-axis public dataset (with images and labels) at the 2017 Automated Cardiac Diagnosis Challenge (ACDC). The relevant qualitative and quantitative results (e.g., dice performance and Hausdorff distance) suggest that the proposed model can achieve superior results over those achieved by the state-of-the-art methods, thus proving that MAE and improved self-attention are more effective and promising for medical image registration tasks. Codes and models are available at https://github.com/XinXiao101/MAE-TransRNet .

MAIRNet: weakly supervised anatomy-aware multimodal articulated image registration network

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Multimodal MR Image Registration Using Weakly Supervised Constrained Affine Network

Residual Aligner Network

MUsculo-Skeleton-Aware (MUSA) deep learning for anatomically guided head-and-neck CT deformable registration

MAFNet: A two-stage multiple attention fusion network for partial-to-partial point cloud registration

Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration

MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation

Weakly-supervised convolutional neural networks for multimodal image registration

MICDIR: Multi-scale Inverse-consistent Deformable Image Registration using UNetMSS with Self-Constructing Graph Latent

Residual Aligner-based Network (RAN): Motion-separable structure for coarse-to-fine discontinuous deformable registration

ACSGRegNet: A Deep Learning-based Framework for Unsupervised Joint Affine and Diffeomorphic Registration of Lumbar Spine CT via Cross- and Self-Attention Fusion

Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

SearchMorph:Multi-scale Correlation Iterative Network for Deformable Registration

Multiscale unsupervised network for deformable image registration

MF-Net: multi-scale feature extraction-integration network for unsupervised deformable registration

MA-VoxelMorph: Multi-scale attention-based VoxelMorph for nonrigid registration of thoracoabdominal CT images

MACG-Net: Multi-axis cross gating network for deformable medical image registration

Weakly-supervised Biomechanically-constrained CT/MRI Registration of the Spine

DART: DEFORMABLE ANATOMY-AWARE REGISTRATION TOOLKIT FOR LUNG CT REGISTRATION WITH KEYPOINTS SUPERVISION

MAE-TransRNet: An improved transformer-ConvNet architecture with masked autoencoder for cardiac MRI registration