Abstract:Multi-modal Magnetic Resonance Imaging (MRI) offers complementary diagnostic information, but some modalities are limited by the long scanning time. To accelerate the whole acquisition process, MRI reconstruction of one modality from highly undersampled k-space data with another fully-sampled reference modality is an efficient solution. However, the misalignment between modalities, which is common in clinic practice, can negatively affect reconstruction quality. Existing deep learning-based methods that account for inter-modality misalignment perform better, but still share two main common limitations: (1) The spatial alignment task is not adaptively integrated with the reconstruction process, resulting in insufficient complementarity between the two tasks; (2) the entire framework has weak interpretability. In this paper, we construct a novel Deep Unfolding Network with Spatial Alignment, termed DUN-SA, to appropriately embed the spatial alignment task into the reconstruction process. Concretely, we derive a novel joint alignment-reconstruction model with a specially designed cross-modal spatial alignment term. By relaxing the model into cross-modal spatial alignment and multi-modal reconstruction tasks, we propose an effective algorithm to solve this model alternatively. Then, we unfold the iterative steps of the proposed algorithm and design corresponding network modules to build DUN-SA with interpretability. Through end-to-end training, we effectively compensate for spatial misalignment using only reconstruction loss, and utilize the progressively aligned reference modality to provide inter-modality prior to improve the reconstruction of the target modality. Comprehensive experiments on three real datasets demonstrate that our method exhibits superior reconstruction performance compared to state-of-the-art methods.

Bispectrum Unbiasing for Dilation-Invariant Multi-Reference Alignment

Bispectrum Unbiasing for Dilation-Invariant Multi-reference Alignment

Sparse Multi-Reference Alignment : Phase Retrieval, Uniform Uncertainty Principles and the Beltway Problem

A Spectral Method for Stable Bispectrum Inversion with Application to Multireference Alignment

Rates of estimation for high-dimensional multi-reference alignment

The generalized method of moments for multi-reference alignment

Minimax-optimal estimation for sparse multi-reference alignment with collision-free signals

The sample complexity of sparse multi-reference alignment and single-particle cryo-electron microscopy

Denoiser-based projections for 2-D super-resolution multi-reference alignment

Concurrent bias correction in hemodynamic data assimilation.

Harmonic Alignment

Joint reconstruction and bias field correction for undersampled MR imaging

DMRA: An Adaptive Line Spectrum Estimation Method through Dynamical Multi-Resolution of Atoms

Sequence adaptive field-imperfection estimation (SAFE): retrospective estimation and correction of $B_1^+$ and $B_0$ inhomogeneities for enhanced MRF quantification

Intensity warping for multisite MRI harmonization

Bayesian Perspective for Orientation Estimation in Cryo-EM and Cryo-ET

System- and Sample-agnostic Isotropic 3D Microscopy by Weakly Physics-informed, Domain-shift-resistant Axial Deblurring

Incorporating Reference Guided Priors into Calibrationless Parallel Imaging Reconstruction

Rotationally invariant image representation for viewing direction classification in cryo-EM

Intrinsic alignment from multiple shear estimates: A first application to data and forecasts for Stage IV

Deep Unfolding Network with Spatial Alignment for multi-modal MRI reconstruction