Abstract:Conventional and electron microscopy visualize structures in the micrometer to nanometer range, and such visualizations contribute decisively to our understanding of biological processes. Due to different factors in recording processes, microscopy images are subject to noise. Especially at their respective resolution limits, a high degree of noise can negatively effect both image interpretation by experts and further automated processing. However, the deteriorating effects of strong noise can be alleviated to a large extend by image enhancement algorithms. Because of the inherent high noise, a requirement for such algorithms is their applicability directly to noisy images or, in the extreme case, to just a single noisy image without a priori noise level information (referred to as blind zero-shot setting). This work investigates blind zero-shot algorithms for microscopy image denoising. The strategies for denoising applied by the investigated approaches include: filtering methods, recent feed-forward neural networks which were amended to be trainable on noisy images, and recent probabilistic generative models. As datasets we consider transmission electron microscopy images including images of SARS-CoV-2 viruses and fluorescence microscopy images. A natural goal of denoising algorithms is to simultaneously reduce noise while preserving the original image features, e.g., the sharpness of structures. However, in practice, a tradeoff between both aspects often has to be found. Our performance evaluations, therefore, focus not only on noise removal but set noise removal in relation to a metric which is instructive about sharpness. For all considered approaches, we numerically investigate their performance, report their denoising/sharpness tradeoff on different images, and discuss future developments. We observe that, depending on the data, the different algorithms can provide significant advantages or disadvantages in terms of their noise removal vs. sharpness preservation capabilities, which may be very relevant for different virological applications, e.g., virological analysis or image segmentation. For any image (e.g., recorded using digital photography or electron microscopy), there is a limiting resolution for which the image still looks sharp. Beyond this resolution (by further zooming in), an image becomes increasingly noisy. Resolution limits may be improved technically (bigger lenses, more light, longer exposure times etc.). Alternatively, or additionally, modern denoising algorithms can be applied after the image is recorded. Here we investigate state-of-the-art such algorithms with many of them being developed in field of Machine Learning. The algorithms work by first learning representations of images (e.g., edges, textures etc.), which they then use to remove the noise from an image. To be applicable at high resolutions, algorithms need to be able to learn from noisy images. If just one noisy image is available for learning without a priori noise level information, the task is also referred to as blind zero-shot denoising. Importantly, we argue that the implicit goal of denoising is noise removal without majorly diminishing image sharpness. All the here investigated blind zero-shot algorithms are consequently evaluated based on both: noise removal and sharpness preservation. We observed salient strengths and weaknesses of different approaches for different image types, which can significantly impact different tasks.

Zero-Shot Noise2Noise: Efficient Image Denoising without any Data

Grouped Multi-Scale Network for Real-World Image Denoising.

A Neural Network Based Low-Light Image Denoising Method

Revealing the Denoising Principle of Zero-Shot N2N-Based Algorithm from 1D Spectrum to 2D Image

Zero-shot denoising of microscopy images recorded at high-resolution limits

Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images

Shot Noise Reduction in Radiographic and Tomographic Multi-Channel Imaging with Self-Supervised Deep Learning

Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training

Neighbor2Neighbor: A Self-Supervised Framework for Deep Image Denoising

Zero-Shot Image Denoising for High-Resolution Electron Microscopy

Masked Pre-trained Model Enables Universal Zero-shot Denoiser

Research Advanced in Image Denoising Based on Deep Learning

Remote Sensing Image Mixed Noise Denoising with Noise Parameter Estimation

Zero-Shot Restoration of Underexposed Images via Robust Retinex Decomposition

Enhancing convolutional neural network generalizability via low-rank weight approximation

Masked Pre-training Enables Universal Zero-shot Denoiser

Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation

Noise2Blur: Online Noise Extraction and Denoising

Dual strategy for single image denoising and generation using deep neural network

Adaptive noise imitation for image denoising