Abstract:Multimodal medical images are widely used by clinicians and physicians to analyze and retrieve complementary information from high-resolution images in a non-invasive manner. Loss of corresponding image resolution adversely affects the overall performance of medical image interpretation. Deep learning-based single image super resolution (SISR) algorithms have revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) with deep multi-attention modules to learn high-frequency information from low-frequency data. Existing approaches based on the GAN have yielded good SR results; however, the texture details of their SR output have been experimentally confirmed to be deficient for medical images particularly. The integration of wavelet transform (WT) and GANs in our proposed SR model addresses the aforementioned limitation concerning textons. While the WT divides the LR image into multiple frequency bands, the transferred GAN uses multi-attention and upsample blocks to predict high-frequency components. Additionally, we present a learning method for training domain-specific classifiers as perceptual loss functions. Using a combination of multi-attention GAN loss and a perceptual loss function results in an efficient and reliable performance. Applying the same model for medical images from diverse modalities is challenging, our work addresses (ii) by training and performing on several modalities via transfer learning. Using two medical datasets, we validate our proposed SR network against existing state-of-the-art approaches and achieve promising results in terms of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).

Multimodal Image Enhancement Using Convolutional Sparse Coding

Convolutional Neural Network Based Computational Imaging Spectroscopy

CSwT-SR: Conv-Swin Transformer for Blind Remote Sensing Image Super-Resolution with Amplitude-Phase Learning and Structural Detail Alternating Learning

Image Super Resolution Based on Salient Sparse Coding

Multimodal-Boost: Multimodal Medical Image Super-Resolution Using Multi-Attention Network With Wavelet Transform

Remote sensing image super-resolution using multi-scale convolutional sparse coding network

Image super resolution using saliency-modulated context-aware sparse decomposition

Multiscale Feature-Learning with a Unified Model for Hyperspectral Image Classification

Single Satellite Imagery Superresolution Based on Hybrid Nonlocal Similarity Constrained Convolution Sparse Coding

Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Coding

SINGLE IMAGE SUPER RESOLUTION VIA COUPLED SPARSE AND LOW RANK DICTIONARY LEARNING

A Wavelet-Based Asymmetric Convolution Network for Single Image Super-Resolution

Dual-branch spectral–spatial feature extraction network for multispectral image compression

MLWAN: Multi-Scale Learning Wavelet Attention Module Network for Image Super Resolution

Image Super-Resolution Via Dual-Dictionary Learning And Sparse Representation

Superpixel-feature-based multiple kernel sparse representation for hyperspectral image classification

Extended wavelet sparse convolutional neural network (EWSCNN) for super resolution

Joint Wavelet Sub-bands Guided Network for Single Image Super-Resolution

Image super-resolution via sparse representation.

End-to-End Multispectral Image Compression Using Convolutional Neural Network

SUPER-RESOLUTION OF MULTISPECTRAL SATELLITE IMAGES USING CONVOLUTIONAL NEURAL NETWORKS