niiv: Fast Self-supervised Neural Implicit Isotropic Volume Reconstruction

Jakob Troidl,Yiqing Liang,Johanna Beyer,Mojtaba Tavakoli,Johann Danzl,Markus Hadwiger,Hanspeter Pfister,James Tompkin

DOI: https://doi.org/10.1101/2024.09.07.611785

2024-09-13

Abstract:Three-dimensional (3D) microscopy data often is anisotropic with significantly lower resolution (up to 8×) along the axis than along the axes. Computationally generating plausible isotropic resolution from anisotropic imaging data would benefit the visual analysis of large-scale volumes. This paper proposes , a self-supervised method for isotropic reconstruction of 3D microscopy data that can quickly produce images at arbitrary (continuous) output resolutions. Within a neural field, the representation embeds a learned latent code that describes the implicit higher-resolution isotropic image region. Under isotropic volume assumptions, we self-supervise this representation on low-/high-resolution lateral image pairs to reconstruct an isotropic volume from low-resolution axial images. We evaluate our method on simulated and real anisotropic electron (EM) and light microscopy (LM) data. Compared to a state-of-the-art diffusion-based method, shows improved reconstruction quality (+1 dB PSNR) and is over three orders of magnitude faster (2,000×) to infer. Specifically, reconstructs a 128 voxel volume in 1/10th of a second, renderable at varying (continuous) high resolutions for display.

Bioinformatics

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper aims to address the common issue of anisotropy in 3D microscopy data. Specifically, such data typically have much lower resolution along the z-axis compared to the xy-plane (up to 8 times lower). This anisotropy limits the ability to perform high-resolution isotropic analysis on large-scale volumetric data.

The paper proposes a self-supervised method named niiv for the rapid reconstruction of isotropic 3D microscopic images from anisotropic imaging data, capable of generating images at any continuous output resolution. Compared to existing diffusion models, niiv improves reconstruction quality (+1 dB PSNR) and increases inference speed by three orders of magnitude (2000 times), requiring only 0.1 seconds to reconstruct a \(128^3\) voxel volume. Additionally, niiv learns higher-resolution isotropic image regions by embedding implicit representations in neural fields and uses low/high-resolution lateral image pairs for self-supervised training under the isotropic volume assumption, thereby reconstructing isotropic volumetric data from low-resolution axial images.

Evaluations on both simulated and real anisotropic data from electron microscopy (EM) and light microscopy (LM) show that niiv outperforms current state-of-the-art diffusion models and other methods in terms of reconstruction quality and speed.