Zero-shot denoising of microscopy images recorded at high-resolution limits
Sebastian Salwig,Jakob Drefs,Jörg Lücke
DOI: https://doi.org/10.1371/journal.pcbi.1012192
2024-06-11
PLoS Computational Biology
Abstract:Conventional and electron microscopy visualize structures in the micrometer to nanometer range, and such visualizations contribute decisively to our understanding of biological processes. Due to different factors in recording processes, microscopy images are subject to noise. Especially at their respective resolution limits, a high degree of noise can negatively effect both image interpretation by experts and further automated processing. However, the deteriorating effects of strong noise can be alleviated to a large extend by image enhancement algorithms. Because of the inherent high noise, a requirement for such algorithms is their applicability directly to noisy images or, in the extreme case, to just a single noisy image without a priori noise level information (referred to as blind zero-shot setting). This work investigates blind zero-shot algorithms for microscopy image denoising. The strategies for denoising applied by the investigated approaches include: filtering methods, recent feed-forward neural networks which were amended to be trainable on noisy images, and recent probabilistic generative models. As datasets we consider transmission electron microscopy images including images of SARS-CoV-2 viruses and fluorescence microscopy images. A natural goal of denoising algorithms is to simultaneously reduce noise while preserving the original image features, e.g., the sharpness of structures. However, in practice, a tradeoff between both aspects often has to be found. Our performance evaluations, therefore, focus not only on noise removal but set noise removal in relation to a metric which is instructive about sharpness. For all considered approaches, we numerically investigate their performance, report their denoising/sharpness tradeoff on different images, and discuss future developments. We observe that, depending on the data, the different algorithms can provide significant advantages or disadvantages in terms of their noise removal vs. sharpness preservation capabilities, which may be very relevant for different virological applications, e.g., virological analysis or image segmentation. For any image (e.g., recorded using digital photography or electron microscopy), there is a limiting resolution for which the image still looks sharp. Beyond this resolution (by further zooming in), an image becomes increasingly noisy. Resolution limits may be improved technically (bigger lenses, more light, longer exposure times etc.). Alternatively, or additionally, modern denoising algorithms can be applied after the image is recorded. Here we investigate state-of-the-art such algorithms with many of them being developed in field of Machine Learning. The algorithms work by first learning representations of images (e.g., edges, textures etc.), which they then use to remove the noise from an image. To be applicable at high resolutions, algorithms need to be able to learn from noisy images. If just one noisy image is available for learning without a priori noise level information, the task is also referred to as blind zero-shot denoising. Importantly, we argue that the implicit goal of denoising is noise removal without majorly diminishing image sharpness. All the here investigated blind zero-shot algorithms are consequently evaluated based on both: noise removal and sharpness preservation. We observed salient strengths and weaknesses of different approaches for different image types, which can significantly impact different tasks.
biochemical research methods,mathematical & computational biology