Abstract:Objective Multimodal medical image fusion (MMIF) technologies merges diverse medical images with rich information, boosting diagnostic efficiency and accuracy. Due to global optimization and single-valued nature, convolutional sparse representation (CSR) outshines the standard sparse representation (SR) in significance. By addressing the challenges of sensitivity to highly redundant dictionaries and robustness to misregistration, an adaptive convolutional sparsity scheme with measurement of the sub-band correlation in the non-subsampled contourlet transform (NSCT) domain is proposed for MMIF. Approach The fusion scheme incorporates four main components: image decomposition into two scales, fusion of detail layers, fusion of base layers, and reconstruction of the two scales. We solved a Tikhonov regularization optimization problem with source images to obtain the base and detail layers. Then, after CSR processing, detail layers were sparsely decomposed using pre-trained dictionary filters for initial coefficient maps. NSCT domain's sub-band correlation was used to refine fusion coefficient maps, and sparse reconstruction produced the fused detail layer. Meanwhile, base layers were fused using averaging. The final fused image was obtained via two-scale reconstruction. Main results Experimental validation of clinical image sets revealed that the proposed fusion scheme can not only effectively eliminate the interference of partial misregistration, but also outperform the representative state-of-the-art fusion schemes in the preservation of structural and textural details according to subjective visual evaluations and objective quality evaluations. Significance The proposed fusion scheme is competitive due to its low-redundancy dictionary, robustness to misregistration, and better fusion performance. This is achieved by training the dictionary with minimal samples through CSR to adaptively preserve overcompleteness for detail layers, and constructing fusion activity level with sub-band correlation in the NSCT domain to maintain CSR attributes. Additionally, ordering the NSCT for reverse sparse representation further enhances sub-band correlation to promote the preservation of structural and textural details.

Feature level MRI fusion based on 3D dual tree compactly supported Shearlet transform

Image Fusion of CT and MR with Sparse Representation in NSST Domain

CT and MRI Image Fusion Based on Multiscale Decomposition Method and Hybrid Approach.

HID: The Hybrid Image Decomposition Model for MRI and CT Fusion

An Efficient Fusion Algorithm Combining Feature Extraction and Variational Optimization for CT and MR Images

Medical Image Fusion Based on Variational and Nonlinear Structure Tensor

High-Quality Fusion and Visualization for MR-PET Brain Tumor Images via Multi-Dimensional Features

A fusion algorithm based on composite decomposition for PET and MRI medical images

Three-Dimensional Medical Image Fusion with Deformable Cross-Attention

CT and MRI image fusion algorithm based on hybrid l 0 l 1 layer decomposing and two-dimensional variation transform

DCT-net: Dual-domain cross-fusion transformer network for MRI reconstruction

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

MRI/CT Fusion Based on Latent Low Rank Representation and Gradient Transfer

CT and MRI Image Fusion Algorithm Based on Hybrid ℓ0ℓ1 Layer Decomposing and Two-Dimensional Variation Transform

Hybrid-Fusion Transformer for Multisequence MRI

An Image Fusion Method Based on Sparse Representation and Sum Modified-Laplacian in NSCT Domain

Multiframe Medical Images Enhancement on Dual Tree Complex Wavelet Transform Domain

Multimodal MRI Volumetric Data Fusion With Convolutional Neural Networks

An advanced MRI and MRSI data fusion scheme for enhancing unsupervised brain tumor differentiation

Medical Image Fusion Based on Feature Extraction and Sparse Representation

TDFusion: When Tensor Decomposition Meets Medical Image Fusion in the Nonsubsampled Shearlet Transform Domain