Abstract:Objective Multimodal medical image fusion (MMIF) technologies merges diverse medical images with rich information, boosting diagnostic efficiency and accuracy. Due to global optimization and single-valued nature, convolutional sparse representation (CSR) outshines the standard sparse representation (SR) in significance. By addressing the challenges of sensitivity to highly redundant dictionaries and robustness to misregistration, an adaptive convolutional sparsity scheme with measurement of the sub-band correlation in the non-subsampled contourlet transform (NSCT) domain is proposed for MMIF. Approach The fusion scheme incorporates four main components: image decomposition into two scales, fusion of detail layers, fusion of base layers, and reconstruction of the two scales. We solved a Tikhonov regularization optimization problem with source images to obtain the base and detail layers. Then, after CSR processing, detail layers were sparsely decomposed using pre-trained dictionary filters for initial coefficient maps. NSCT domain's sub-band correlation was used to refine fusion coefficient maps, and sparse reconstruction produced the fused detail layer. Meanwhile, base layers were fused using averaging. The final fused image was obtained via two-scale reconstruction. Main results Experimental validation of clinical image sets revealed that the proposed fusion scheme can not only effectively eliminate the interference of partial misregistration, but also outperform the representative state-of-the-art fusion schemes in the preservation of structural and textural details according to subjective visual evaluations and objective quality evaluations. Significance The proposed fusion scheme is competitive due to its low-redundancy dictionary, robustness to misregistration, and better fusion performance. This is achieved by training the dictionary with minimal samples through CSR to adaptively preserve overcompleteness for detail layers, and constructing fusion activity level with sub-band correlation in the NSCT domain to maintain CSR attributes. Additionally, ordering the NSCT for reverse sparse representation further enhances sub-band correlation to promote the preservation of structural and textural details.

Multimodal medical image fusion with convolution sparse representation and mutual information correlation in NSST domain

Image Fusion of CT and MR with Sparse Representation in NSST Domain

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

An Image Fusion Method Based on Sparse Representation and Sum Modified-Laplacian in NSCT Domain

Multimodal medical image fusion in NSST domain with structural and spectral features enhancement

Medical Image Fusion with Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain

Research of Multimodal Medical Image Fusion Based on Parameter-Adaptive Pulse-Coupled Neural Network and Convolutional Sparse Representation

Medical Image Fusion Method Based on Coupled Neural P Systems in Nonsubsampled Shearlet Transform Domain

Feature-Motivated Simplified Adaptive PCNN-Based Medical Image Fusion Algorithm in NSST Domain

Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain

Medical Image Fusion Via Convolutional Sparsity Based Morphological Component Analysis

Log-Gabor energy based multimodal medical image fusion in NSCT domain

Multi-modal medical image fusion using improved dual-channel PCNN

Medical image fusion based on sparse representation and neighbor energy activity

Multimodal medical image fusion algorithm based on pulse coupled neural networks and nonsubsampled contourlet transform

Multimodal medical image fusion based on visual saliency map and multichannel dynamic threshold neural P systems in sub-window variance filter domain

A Novel Method of Multimodal Medical Image Fusion Based on Hybrid Approach of NSCT and DTCWT

Multimodality Medical Image Fusion Using Clustered Dictionary Learning in Non-Subsampled Shearlet Transform

NSCT-DCT based Fourier Analysis for Fusion of Multimodal Images

Multistage multimodal medical image fusion model using feature‐adaptive pulse coupled neural network

A Medical Image Fusion Method Based on SIFT and Deep Convolutional Neural Network in the SIST Domain