Abstract:High-resolution microscopy hyperspectral (HS) images can provide highly detailed spatial and spectral information, enabling the identification and analysis of biological tissues at a microscale level. Recently, significant efforts have been devoted to enhancing the resolution of HS images by leveraging high spatial resolution multispectral (MS) images. However, the inherent hardware constraints lead to a significant distribution gap between HS and MS images, posing challenges for image super-resolution within biomedical domains. This discrepancy may arise from various factors, including variations in camera imaging principles (e.g., snapshot and push-broom imaging), shooting positions, and the presence of noise interference. To address these challenges, we introduced a unique unsupervised super-resolution framework named R2D2-GAN. This framework utilizes a generative adversarial network (GAN) to efficiently merge the two data modalities and improve the resolution of microscopy HS images. Traditionally, supervised approaches have relied on intuitive and sensitive loss functions, such as mean squared error (MSE). Our method, trained in a real-world unsupervised setting, benefits from exploiting consistent information across the two modalities. It employs a game-theoretic strategy and dynamic adversarial loss, rather than relying solely on fixed training strategies for reconstruction loss. Furthermore, we have augmented our proposed model with a central consistency regularization (CCR) module, aiming to further enhance the robustness of the R2D2-GAN. Our experimental results show that the proposed method is accurate and robust for super-resolution images. We specifically tested our proposed method on both a real and a synthetic dataset, obtaining promising results in comparison to other state-of-the-art methods. Our code and datasets are accessible through Multimedia Content.

A Cascaded Refinement GAN for Phase Contrast Microscopy Image Super Resolution.

Infrared Image Super Resolution Using Gan With Infrared Image Prior

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Super-resolution reconstruction method of the optical synthetic aperture image using generative adversarial network

R2D2-GAN: Robust Dual Discriminator Generative Adversarial Network for Microscopy Hyperspectral Image Super-Resolution

Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy

High-throughput, high-resolution registration-free generated adversarial network microscopy

Resolution enhancement and realistic speckle recovery with generative adversarial modeling of micro-optical coherence tomography

G-GANISR: Gradual generative adversarial network for image super resolution

Super-Resolution Reconstruction of Cell Images Based on Generative Adversarial Networks

Retinal fundus image superresolution generated by optical coherence tomography based on a realistic mixed attention GAN

A Residual Dense Attention Generative Adversarial Network for Microscopic Image Super-Resolution

Progressive Generative Adversarial Networks for Medical Image Super resolution

Multi-Frame Super-Resolution Algorithm Based on a WGAN

Image super-resolution using progressive generative adversarial networks for medical image analysis

Restoration of X-ray phase-contrast imaging based on generative adversarial networks

SMC-SRGAN-Lightning super-resolution algorithm based on optical micro-scanning thermal microscope image

High-resolution medical image synthesis using progressively grown generative adversarial networks

Frequency-aware optical coherence tomography image super-resolution via conditional generative adversarial neural network

Csrgan: medical image super-resolution using a generative adversarial network

Research on Image Denoising and Super-Resolution Reconstruction Technology of Multiscale-Fusion Images