DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images

Mingjie Pan,Yulu Gan,Fangxu Zhou,Jiaming Liu,Aimin Wang,Shanghang Zhang,Dawei Li

2023-06-21

Abstract:Three-dimensional microscopy is often limited by anisotropic spatial resolution, resulting in lower axial resolution than lateral resolution. Current State-of-The-Art (SoTA) isotropic reconstruction methods utilizing deep neural networks can achieve impressive super-resolution performance in fixed imaging settings. However, their generality in practical use is limited by degraded performance caused by artifacts and blurring when facing unseen anisotropic factors. To address these issues, we propose DiffuseIR, an unsupervised method for isotropic reconstruction based on diffusion models. First, we pre-train a diffusion model to learn the structural distribution of biological tissue from lateral microscopic images, resulting in generating naturally high-resolution images. Then we use low-axial-resolution microscopy images to condition the generation process of the diffusion model and generate high-axial-resolution reconstruction results. Since the diffusion model learns the universal structural distribution of biological tissues, which is independent of the axial resolution, DiffuseIR can reconstruct authentic images with unseen low-axial resolutions into a high-axial resolution without requiring re-training. The proposed DiffuseIR achieves SoTA performance in experiments on EM data and can even compete with supervised methods.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in three - dimensional microscopic imaging, due to the limitation of physical slice intervals or the consideration of saving time, the axial (z - direction) resolution is lower than the lateral (xy - plane) resolution. This anisotropic resolution limits the ability to reconstruct high - quality 3D images. Although existing deep - learning - based methods can achieve impressive super - resolution performance under fixed imaging settings, their performance degrades due to artifacts and blurring when facing unseen anisotropic factors, which limits their universality in practical applications. To overcome these problems, the paper proposes an unsupervised method named DiffuseIR, which is based on the diffusion model to achieve isotropic reconstruction. Specifically, DiffuseIR first pre - trains a diffusion model to make it learn the structural distribution of biological tissues from high - resolution lateral microscopic images, and then uses low - axial - resolution microscopic images to adjust the generation process of the diffusion model, thereby generating high - axial - resolution reconstruction results. Since the diffusion model learns the general structural distribution of biological tissues that is independent of axial resolution, DiffuseIR can reconstruct images with unseen low - axial - resolution into high - axial - resolution images without retraining, showing stronger robustness and generalization ability.

DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images

DiffuseIR: Diffusion Models for Isotropic Reconstruction of 3D Microscopic Images

MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections

niiv: Fast Self-supervised Neural Implicit Isotropic Volume Reconstruction

Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior

Self-Supervised Super-Resolution Approach for Isotropic Reconstruction of 3D Electron Microscopy Images from Anisotropic Acquisition

From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task

Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models

Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models

Diffusion-based deep learning method for augmenting ultrastructural imaging and volume electron microscopy

Isotropic Reconstruction of 3D EM Images with Unsupervised Degradation Learning

PrideDiff: Physics-Regularized Generalized Diffusion Model for CT Reconstruction

Image Reconstruction for Diffuse Optical Tomography Based on Radiative Transfer Equation

System- and Sample-agnostic Isotropic 3D Microscopy by Weakly Physics-informed, Domain-shift-resistant Axial Deblurring

Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution

TC-DiffRecon: Texture coordination MRI reconstruction method based on diffusion model and modified MF-UNet method

Reconstructing 3D Biomedical Architectural Order at Multiple Spatial Scales with Multimodal Stack Input

Reconstruction of Isotropic High-Resolution MR Image from Multiple Anisotropic Scans Using Sparse Fidelity Loss and Adversarial Regularization

Resolution-Robust 3D MRI Reconstruction with 2D Diffusion Priors: Diverse-Resolution Training Outperforms Interpolation

Deep Self-Learning Enables Fast, High-Fidelity Isotropic Resolution Restoration for Volumetric Fluorescence Microscopy.

DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction