Noise-Resilient Ptychographic Imaging with Deep Learning

Han Yue,Yun Xie,Xin Lou,Jun Cheng,Yu-Xuan Ren,Feng Shu

DOI: https://doi.org/10.1109/cleo-pr60912.2024.10676456

2024-01-01

Abstract:We develop Ptyde-Net for enhancing the SNR in ptychographic phase retrieval in noisy environments, with specially designed denoising modules and attention mechanisms, which are evidenced by remarkable improvements in both PSNR and SSIM. © 2024 The Author(s)

What problem does this paper attempt to address?

Single-frame Noisy Interferogram Phase Retrieval Using an End-to-end Deep Learning Network with Physical Information Constraints

Tian Zhang,Runzhou Shi,Yuqi Shao,Qijie Chen,Jian Bai

DOI: https://doi.org/10.1016/j.optlaseng.2024.108419

IF: 5.666

2024-01-01

Optics and Lasers in Engineering

Abstract:Phase retrieval from a single-frame interferogram is a meaningful approach in dynamic and real-time interferometry. However, existing demodulation methods are susceptible to real noise impressions, resulting in low accuracy. This paper proposes an accurate end-to-end learning-based method for one-shot phase retrieval that doesn't require pre-processing (e.g., denoising, normalization) and post-processing (e.g., phase unwrapping). Compared to existing deep learning methods, we incorporate more physical information constraints, contributing to high accuracy and robustness. A realistic noise model that incorporates the speckle noise and the imaging process is introduced to generate training datasets. Furthermore, a subnetwork is adopted for noise estimation, along with a subnetwork for frequency domain decomposition to extract multi-channel physical information. Additionally, a physical information fusion block is established to integrate relevant data, and a reconstruction network based on NAFNet is implemented for comprehensive end-to-end phase retrieval. We validate the accuracy and robustness of our approach through simulations and experiments. Our results demonstrate that the proposed method achieves a phase retrieval accuracy of 0.01 lambda for noisy interferograms, superior to several existing methods that we have reproduced.
Frequency-Relevant Residual Learning for Multi-Modal Image Denoising.

Xiongwei Liu,Zehua Sheng,Hui-Liang Shen

DOI: https://doi.org/10.1109/icip46576.2022.9898074

2022-01-01

Abstract:Recently, multi-modal image processing has shown its great potential in boosting denoising performance in terms of both accuracy and visual quality. However, current studies generally face the challenge of achieving a good balance between noise removal and detail preservation. In this work, we introduce patch-wise frequency decomposition into convolutional neural networks and propose a novel multi-modal image denoising (MID) algorithm. Integrating the frequency-domain information of images from different modalities, our network first predicts a frequency-relevant residual and then regresses the denoised result using a learnable reconstruction kernel. Benefiting from the distinctive properties of noise and true signals as well as the correlation between multi-modal images in the frequency domain, the proposed algorithm can effectively remove noise and simultaneously reconstruct fine details. Extensive experiments demonstrate the superiority and generalizability of our algorithm over state-of-the-art competing algorithms on various MID tasks, including near-infrared guided RGB image denoising, flash guided no-flash image denoising, and RGB guided depth image denoising. Code is available at https://github.com/liuxw11/FRL.
Ptychoformer: A Physics-Guided Deep Learning Framework for Ptychographic Imaging

Han Yue,Jun Cheng,Yu-Xuan Ren,Philip Heng Wai Leong,Steve Feng Shu

2024-11-25

Abstract:Ptychographic imaging confronts limitations in applying deep learning (DL) for retrieval from diffraction patterns. Conventional neural architectures are optimized for natural images, overlooking the unique physical characteristics of diffraction data, including radial intensity decay and coherent information distributed in concentric rings. In this paper, we present Ptychoformer, a physics-guided DL framework for ptychographic imaging that aligns attention mechanisms and feature extraction with these diffraction physics properties through introducing a dual-branch architecture which accounts for both local and non-local dependencies from the patterns. It consists of a Polar Coordinate Attention (PCA) mechanism that is inspired by the Ewald construction in X-ray crystallography to enhance high-frequency component fidelity. Experimental results demonstrate Ptychoformer's superior performance across both simulated and real data in preserving fine details and suppressing artifacts. On simulated data, Ptychoformer achieves up to 5.4% higher PSNR and 4.2% higher SSIM for amplitude retrieval compared to existing methods. For real experimental data, it demonstrates up to 12.5% higher PSNR and 31.3% higher SSIM for amplitude retrieval. Notably, Ptychoformer maintains robust performance under limited training data and low overlap ratios, outperforming existing models.

Optics,Computer Vision and Pattern Recognition
Ptychographic phase retrieval via a deep‐learning‐assisted iterative algorithm

Koki Yamada,Natsuki Akaishi,Kohei Yatabe,Yuki Takayama

DOI: https://doi.org/10.1107/s1600576724006897

IF: 4.868

2024-08-22

Journal of Applied Crystallography

Abstract:This paper proposes a ptychographic phase‐retrieval algorithm combined with a deep neural network (DNN). The proposed method allows the measurement model to be explicitly incorporated into the DNN‐based approach, improving imaging capability and robustness to changes in experimental conditions.Ptychography is a powerful computational imaging technique with microscopic imaging capability and adaptability to various specimens. To obtain an imaging result, it requires a phase‐retrieval algorithm whose performance directly determines the imaging quality. Recently, deep neural network (DNN)‐based phase retrieval has been proposed to improve the imaging quality from the ordinary model‐based iterative algorithms. However, the DNN‐based methods have some limitations because of the sensitivity to changes in experimental conditions and the difficulty of collecting enough measured specimen images for training the DNN. To overcome these limitations, a ptychographic phase‐retrieval algorithm that combines model‐based and DNN‐based approaches is proposed. This method exploits a DNN‐based denoiser to assist an iterative algorithm like ePIE in finding better reconstruction images. This combination of DNN and iterative algorithms allows the measurement model to be explicitly incorporated into the DNN‐based approach, improving its robustness to changes in experimental conditions. Furthermore, to circumvent the difficulty of collecting the training data, it is proposed that the DNN‐based denoiser be trained without using actual measured specimen images but using a formula‐driven supervised approach that systemically generates synthetic images. In experiments using simulation based on a hard X‐ray ptychographic measurement system, the imaging capability of the proposed method was evaluated by comparing it with ePIE and rPIE. These results demonstrated that the proposed method was able to reconstruct higher‐spatial‐resolution images with half the number of iterations required by ePIE and rPIE, even for data with low illumination intensity. Also, the proposed method was shown to be robust to its hyperparameters. In addition, the proposed method was applied to ptychographic datasets of a Simens star chart and ink toner particles measured at SPring‐8 BL24XU, which confirmed that it can successfully reconstruct images from measurement scans with a lower overlap ratio of the illumination regions than is required by ePIE and rPIE.

chemistry, multidisciplinary,crystallography
Noise-robust latent vector reconstruction in ptychography using deep generative models

Jacob Seifert,Yifeng Shao,Allard P. Mosk

DOI: https://doi.org/10.1364/OE.513556

2024-01-15

Abstract:Computational imaging is increasingly vital for a broad spectrum of applications, ranging from biological to material sciences. This includes applications where the object is known and sufficiently sparse, allowing it to be described with a reduced number of parameters. When no explicit parameterization is available, a deep generative model can be trained to represent an object in a low-dimensional latent space. In this paper, we harness this dimensionality reduction capability of autoencoders to search for the object solution within the latent space rather than the object space. We demonstrate a novel approach to ptychographic image reconstruction by integrating a deep generative model obtained from a pre-trained autoencoder within an Automatic Differentiation Ptychography (ADP) framework. This approach enables the retrieval of objects from highly ill-posed diffraction patterns, offering an effective method for noise-robust latent vector reconstruction in ptychography. Moreover, the mapping into a low-dimensional latent space allows us to visualize the optimization landscape, which provides insight into the convexity and convergence behavior of the inverse problem. With this work, we aim to facilitate new applications for sparse computational imaging such as when low radiation doses or rapid reconstructions are essential.

Image and Video Processing,Computational Physics,Optics
An adaptive noise-blind-separation algorithm for ptychography

Li Liu,Wenjie Li,Lei Zhong,Honggang Gu,Shiyuan Liu

DOI: https://doi.org/10.1016/j.optlaseng.2023.107748

IF: 5.666

2023-07-29

Optics and Lasers in Engineering

Abstract:Noise, one of the severe challenges in phase retrieval, may bring the algorithms to fall into local minima or even disrupt the convergence, thereby adversely affecting the imaging quality and convergence efficiency in the ptychography. Here, we propose an adaptive noise-blind-separation (aNBS) algorithm to deal with mixed noises of different types without sensitivity discrepancy in ptychography. In the aNBS algorithm, apart from the coherent probe of the illumination wavelength, virtual probes of different wavelengths are introduced to capture the noise energies, and the coupling mixed noises are adaptively blind-separated into the virtual noise probes in ptychography. The proposed algorithm can blindly separate coherent diffraction signals and noises without requiring additional regularization constraints, noise prior assumptions or dynamic iteration parameters. Simulations and experiments are comparatively conducted with the momentum-accelerated ptychographical iterative engine algorithm without dealing with noises and the least-square maximum-likelihood ptychographical iterative engine algorithm using a mixed Poisson-Gaussian likelihood model. Results indicate that the aNBS-based ptychography significantly enhances the convergence robustness when the noise intensity increases by more than two orders of magnitude. Compared with existing methods, the proposed aNBS algorithm demonstrates superior robustness and generality for the phase retrieval in coherent diffraction imaging and can be widely applied to various fields, such as ptychography, holography, and tomography.

optics
Unsupervised denoising of photoacoustic images based on the Noise2Noise network

Yanda Cheng,Wenhan Zheng,Robert Bing,Huijuan Zhang,Chuqin Huang,Peizhou Huang,Leslie Ying,Jun Xia

DOI: https://doi.org/10.1364/BOE.529253

2024-07-02

Abstract:In this study, we implemented an unsupervised deep learning method, the Noise2Noise network, for the improvement of linear-array-based photoacoustic (PA) imaging. Unlike supervised learning, which requires a noise-free ground truth, the Noise2Noise network can learn noise patterns from a pair of noisy images. This is particularly important for in vivo PA imaging, where the ground truth is not available. In this study, we developed a method to generate noise pairs from a single set of PA images and verified our approach through simulation and experimental studies. Our results reveal that the method can effectively remove noise, improve signal-to-noise ratio, and enhance vascular structures at deeper depths. The denoised images show clear and detailed vascular structure at different depths, providing valuable insights for preclinical research and potential clinical applications.
Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising

Hansen Feng,Lizhi Wang,Yiqi Huang,Yuzhi Wang,Lin Zhu,Hua Huang

2024-01-22

Abstract:Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distribution, while learning-based noise modeling impractically depends on paired real data. In this paper, we propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency. Based on this strategy, we introduce an efficient physics-guided noise neural proxy (PNNP) to approximate the real-world sensor noise model. Specifically, we integrate physical priors into neural proxies and introduce three efficient techniques: physics-guided noise decoupling (PND), physics-guided proxy model (PPM), and differentiable distribution loss (DDL). PND decouples the dark frame into different components and handles different levels of noise flexibly, which reduces the complexity of noise modeling. PPM incorporates physical priors to constrain the generated noise, which promotes the accuracy of noise modeling. DDL provides explicit and reliable supervision for noise distribution, which promotes the precision of noise modeling. PNNP exhibits powerful potential in characterizing the real noise distribution. Extensive experiments on public datasets demonstrate superior performance in practical low-light raw image denoising. The code will be available at \url{<a class="link-external link-https" href="https://github.com/fenghansen/PNNP" rel="external noopener nofollow">this https URL</a>}.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
Addressing image and Poisson noise deconvolution problem using deep learning approaches

Mohammad Haider Syed,Kamal Upreti,Mohammad Shahnawaz Nasir,Mohammad Shabbir Alam,Arvind Sharma

DOI: https://doi.org/10.1111/coin.12510

2022-02-17

Computational Intelligence

Abstract:Abstract Digital images are more important in numerous contemporary applications, and the need for images in the technical field is also increasing drastically. It is used to recognize signatures and faces in many industries and is applicable for intelligent departments. The images are usually associated with the noise content; this may happen due to the instrument imperfections, troubleshooting while collecting data from the acquisition process, and another natural phenomenon. Poisson noise, also known as photon noise, is caused in the images due to the statistical essence of electromagnetic waves. X‐ray, visible light, and gamma rays are electromagnetic waves. The enhancement of the convolution model in addressing images is challenging due to the various constituents such as optical aberrations, noise level, and optical setup. The modeling configuration of the image is attained using the point spread function (PSF), which is responsible for the system's impulse response. The quality image is retrieved by denoising and super‐resolution (SR) methods; these methods simultaneously eliminate the noise content from the images. A Richardson–Lucy and alternating direction method of multipliers type of non‐blind iterative algorithmic approaches associated with the PSF performance in addressing image is comparatively analyzed. The deep learning approach, convolutional neural networks (CNNs), is also employed to understand the nonlinear mapping relationship between the observed data and ground reality. The performance of the various network approaches is compared in this article. The result obtained shows that the deep learning CNNs achieved higher accuracy in producing denoising images. The goal of the proposed system model is to remove the interference noise in images. The high‐resolution images are obtained by implementing a SR‐based CNN model.

computer science, artificial intelligence
Noise Reduction in Optical Coherence Tomography Images Using a Deep Neural Network with Perceptually-Sensitive Loss Function.

Bin Qiu,Zhiyu Huang,Xi Liu,Xiangxi Meng,Yunfei You,Gangjun Liu,Kun Yang,Andreas Maier,Qiushi Ren,Yanye Lu

DOI: https://doi.org/10.1364/boe.379551

IF: 3.4

2020-01-01

Biomedical Optics Express

Abstract:Optical coherence tomography (OCT) is susceptible to the coherent noise, which is the speckle noise that deteriorates contrast and the detail structural information of OCT images, thus imposing significant limitations on the diagnostic capability of OCT. In this paper, we propose a novel OCT image denoising method by using an end-to-end deep learning network with a perceptually-sensitive loss function. The method has been validated on OCT images acquired from healthy volunteers' eyes. The label images for training and evaluating OCT denoising deep learning models are images generated by averaging 50 frames of respective registered B-scans acquired from a region with scans occurring in one direction. The results showed that the new approach can outperform other related denoising methods on the aspects of preserving detail structure information of retinal layers and improving the perceptual metrics in the human visual perception.
Remote Sensing Image Mixed Noise Denoising with Noise Parameter Estimation

Mutian Wang,Sijie Zhao,Xun Cao,Tao Yue,Xuemei Hu

DOI: https://doi.org/10.1007/978-3-030-27300-2_33

2020-01-01

Abstract:Images acquired by optical remote sensing often suffer from noise, due to low-light conditions and long distances from the object. Noise introduces undesirable effects such as artifacts, unrealistic edges, unseen lines and corners, blurred objects and disturbed background scenes, and decreasing the performance of visual and computational analysis. The great majority of denoising methods only focus on single noise type, and mostly assume additive Gaussian white noise. However, in practice, according to the different sources of noise during the capturing process, the real image noise may be composed of noises from several different distributions, i.e., Poisson distribution from shot noise and dark current, Gaussian distribution from readout noise, etc. Thus, it is hard to model complex noise with a single noise model, which therefore limits the practical performance of image denoiser for real captured images. To address this problem, we propose a deep residual network with parameter estimation to model and remove mixed noises of remote sensing data, with no noise type and noise level information. In this chapter, a deep convolutional neural network based framework is proposed to implement both noise estimation and noise removing. The framework is composed of noise estimator module and denoiser module. The noise estimator module is used to estimate the noise distribution characteristics of the input image, which are used to synthesize the noisy dataset for the training of the denoiser module. After training, the denoiser module can then denoise the input image specifically for that noise characteristic of the input image. Besides, we use some advanced designment to improve the performance of the network like residual learning, skip connections, and perceptual loss. Our model works for both given noise types and others without taking into account in training process, and for the latter case we test the model on the noise mixed with different types, i.e., shot noise, Gaussian noise, salt and pepper noise, speckle noise, dark current, and quantization noise, covering most of the common noise types in practical situation. The estimated noise distribution according to the proposed estimator is compared with the ground truth to validate the performance of the estimator. We also compare the proposed method with the existing denoising methods, including both the traditional algorithm (i.e., BM3D (Dabov et al , IEEE Trans Image Process 16:2080-2095, 2007)) and the deep neural network (i.e., DnCNN (Zhang et al., IEEE Trans Image Process 26:3142-3155, 2017)). Experimental results showed that our method outperforms these state-of-the-art methods in both PSNR (peak signal to noise ratio) and SSIM (structural similarity index).
3D-2D Neural Nets for Phase Retrieval in Noisy Interferometric Imaging

Andrew H. Proppe,Guillaume Thekkadath,Duncan England,Philip J. Bustard,Frédéric Bouchard,Jeff S. Lundeen,Benjamin J. Sussman

2024-02-09

Abstract:In recent years, neural networks have been used to solve phase retrieval problems in imaging with superior accuracy and speed than traditional techniques, especially in the presence of noise. However, in the context of interferometric imaging, phase noise has been largely unaddressed by existing neural network architectures. Such noise arises naturally in an interferometer due to mechanical instabilities or atmospheric turbulence, limiting measurement acquisition times and posing a challenge in scenarios with limited light intensity, such as remote sensing. Here, we introduce a 3D-2D Phase Retrieval U-Net (PRUNe) that takes noisy and randomly phase-shifted interferograms as inputs, and outputs a single 2D phase image. A 3D downsampling convolutional encoder captures correlations within and between frames to produce a 2D latent space, which is upsampled by a 2D decoder into a phase image. We test our model against a state-of-the-art singular value decomposition algorithm and find PRUNe reconstructions consistently show more accurate and smooth reconstructions, with a x2.5 - 4 lower mean squared error at multiple signal-to-noise ratios for interferograms with low (< 1 photon/pixel) and high (~100 photons/pixel) signal intensity. Our model presents a faster and more accurate approach to perform phase retrieval in extremely low light intensity interferometry in presence of phase noise, and will find application in other multi-frame noisy imaging techniques.

Optics,Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
Background Denoising for Ptychography via Wigner Distribution Deconvolution

Oleh Melnyk,Patricia Römer

2024-02-23

Abstract:Ptychography is a computational imaging technique that aims to reconstruct the object of interest from a set of diffraction patterns. Each of these is obtained by a localized illumination of the object, which is shifted after each illumination to cover its whole domain. As in the resulting measurements the phase information is lost, ptychography gives rise to solving a phase retrieval problem. In this work, we consider ptychographic measurements corrupted with background noise, a type of additive noise that is independent of the shift, i.e., it is the same for all diffraction patterns. Two algorithms are provided, for arbitrary objects and for so-called phase objects that do not absorb the light but only scatter it. For the second type, a uniqueness of reconstruction is established for almost every object. Our approach is based on the Wigner Distribution Deconvolution, which lifts the object to a higher-dimensional matrix space where the recovery can be reformulated as a linear problem. Background noise only affects a few equations of the linear system that are therefore discarded. The lost information is then restored using redundancy in the higher-dimensional space.

Image and Video Processing,Information Theory,Numerical Analysis
AI-enabled high-resolution scanning coherent diffraction imaging

Mathew J. Cherukara,Tao Zhou,Youssef Nashed,Pablo Enfedaque,Alex Hexemer,Ross J. Harder,Martin V. Holt

DOI: https://doi.org/10.1063/5.0013065

IF: 4

2020-07-27

Applied Physics Letters

Abstract:Ptychographic imaging is a powerful means of imaging beyond the resolution limits of typical x-ray optics. Recovering images from raw ptychographic data, however, requires the solution of an inverse problem, namely, phase retrieval. Phase retrieval algorithms are computationally expensive, which precludes real-time imaging. In this work, we propose PtychoNN, an approach to solve the ptychography data inversion problem based on a deep convolutional neural network. We demonstrate how the proposed method can be used to predict real-space structure and phase at each scan point solely from the corresponding far-field diffraction data. Our results demonstrate the practical application of machine learning to recover high fidelity amplitude and phase contrast images of a real sample hundreds of times faster than current ptychography reconstruction packages. Furthermore, by overcoming the constraints of iterative model-based methods, we can significantly relax sampling constraints on data acquisition while still producing an excellent image of the sample. Besides drastically accelerating acquisition and analysis, this capability has profound implications for the imaging of dose sensitive, dynamic, and extremely voluminous samples.

physics, applied
Advanced Denoising for X-ray Ptychography

Huibin Chang,Pablo Enfedaque,Jie Zhang,Juliane Reinhardt,Bjoern Enders,Young-Sang Yu,David Shapiro,Christian G. Schroer,Tieyong Zeng,Stefano Marchesini

DOI: https://doi.org/10.1364/OE.27.010395

2019-02-15

Abstract:The success of ptychographic imaging experiments strongly depends on achieving high signal-to-noise ratio. This is particularly important in nanoscale imaging experiments when diffraction signals are very weak and the experiments are accompanied by significant parasitic scattering (background), outliers or correlated noise sources. It is also critical when rare events such as cosmic rays, or bad frames caused by electronic glitches or shutter timing malfunction take place. In this paper, we propose a novel iterative algorithm with rigorous analysis that exploits the direct forward model for parasitic noise and sample smoothness to achieve a thorough characterization and removal of structured and random noise. We present a formal description of the proposed algorithm and prove its convergence under mild conditions. Numerical experiments from simulations and real data (both soft and hard X-ray beamlines) demonstrate that the proposed algorithms produce better results when compared to state-of-the-art methods.

Image and Video Processing,Optimization and Control,Optics
Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training

Yuanhao Cai,Xiaowan Hu,Haoqian Wang,Yulun Zhang,Hanspeter Pfister,Donglai Wei

DOI: https://doi.org/10.48550/arXiv.2204.02844

2022-04-06

Computer Vision and Pattern Recognition

Abstract:Existing deep learning real denoising methods require a large amount of noisy-clean image pairs for supervision. Nonetheless, capturing a real noisy-clean dataset is an unacceptable expensive and cumbersome procedure. To alleviate this problem, this work investigates how to generate realistic noisy images. Firstly, we formulate a simple yet reasonable noise model that treats each real noisy pixel as a random variable. This model splits the noisy image generation problem into two sub-problems: image domain alignment and noise domain alignment. Subsequently, we propose a novel framework, namely Pixel-level Noise-aware Generative Adversarial Network (PNGAN). PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space to perform image domain alignment. Simultaneously, PNGAN establishes a pixel-level adversarial training to conduct noise domain alignment. Additionally, for better noise fitting, we present an efficient architecture Simple Multi-scale Network (SMNet) as the generator. Qualitative validation shows that noise generated by PNGAN is highly similar to real noise in terms of intensity and distribution. Quantitative experiments demonstrate that a series of denoisers trained with the generated noisy images achieve state-of-the-art (SOTA) results on four real denoising benchmarks. Part of codes, pre-trained models, and results are available at https://github.com/caiyuanhao1998/PNGAN for comparisons.
Deep learning optical image denoising research based on principal component estimation

Qianbo Lu,Chengxiu Liu,Wenlu Feng,Qingxiong Xiao,Xiaoxu Wang

DOI: https://doi.org/10.1364/AO.455849

2022-05-20

Abstract:High-quality denoising of optical interference images usually requires preliminary prediction of the noise level. Although blind denoising can filter the image at the pixel level without noise prediction, it inevitably loses a significant amount of phase information. This paper proposes a fast and high-quality denoising algorithm for optical interference images that combines the merits of a principal component analysis (PCA) and residual neural networks. The PCA is used to analyze the image noise and, in turn, establishes an accurate mapping between the estimated and true noise levels. The mapping helps to select a suitable residual neural network model for image processing, which maximizes the retention of image information and reduces the effect of noise. In addition, a comprehensive evaluation factor to account for the time complexity and denoising effect of the algorithm is proposed, since time complexity can be a dominant concern in some cases of actual measurement. The performance of the denoising algorithm and the effectiveness of the evaluation criterion are demonstrated to be high by processing a set of optical interference images and benchmarking with other denoising algorithms. The proposed algorithm outperforms the previously reported counterparts in a specific area of optical interference image preprocessing and provides an alternative paradigm for other denoising problems of optics, such as holograms and structured light measurements.
Physics-based Noise Modeling for Extreme Low-light Photography

Kaixuan Wei,Ying Fu,Yinqiang Zheng,Jiaolong Yang

DOI: https://doi.org/10.48550/arXiv.2108.02158

2021-08-05

Abstract:Enhancing the visibility in extreme low-light environments is a challenging task. Under nearly lightless condition, existing image denoising methods could easily break down due to significantly low SNR. In this paper, we systematically study the noise statistics in the imaging pipeline of CMOS photosensors, and formulate a comprehensive noise model that can accurately characterize the real noise structures. Our novel model considers the noise sources caused by digital camera electronics which are largely overlooked by existing methods yet have significant influence on raw measurement in the dark. It provides a way to decouple the intricate noise structure into different statistical distributions with physical interpretations. Moreover, our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms. In this regard, although promising results have been shown recently with deep convolutional neural networks, the success heavily depends on abundant noisy clean image pairs for training, which are tremendously difficult to obtain in practice. Generalizing their trained models to images from new devices is also problematic. Extensive experiments on multiple low-light denoising datasets -- including a newly collected one in this work covering various devices -- show that a deep neural network trained with our proposed noise formation model can reach surprisingly-high accuracy. The results are on par with or sometimes even outperform training with paired real data, opening a new door to real-world extreme low-light photography.

Image and Video Processing,Computer Vision and Pattern Recognition
De-Noising of Photoacoustic Microscopy Images by Deep Learning

Da He,Jiasheng Zhou,Xiaoyu Shang,Jiajia Luo,Sung-Liang Chen

DOI: https://doi.org/10.48550/arXiv.2201.04302

2022-01-12

Image and Video Processing

Abstract:As a hybrid imaging technology, photoacoustic microscopy (PAM) imaging suffers from noise due to the maximum permissible exposure of laser intensity, attenuation of ultrasound in the tissue, and the inherent noise of the transducer. De-noising is a post-processing method to reduce noise, and PAM image quality can be recovered. However, previous de-noising techniques usually heavily rely on mathematical priors as well as manually selected parameters, resulting in unsatisfactory and slow de-noising performance for different noisy images, which greatly hinders practical and clinical applications. In this work, we propose a deep learning-based method to remove complex noise from PAM images without mathematical priors and manual selection of settings for different input images. An attention enhanced generative adversarial network is used to extract image features and remove various noises. The proposed method is demonstrated on both synthetic and real datasets, including phantom (leaf veins) and in vivo (mouse ear blood vessels and zebrafish pigment) experiments. The results show that compared with previous PAM de-noising methods, our method exhibits good performance in recovering images qualitatively and quantitatively. In addition, the de-noising speed of 0.016 s is achieved for an image with $256\times256$ pixels. Our approach is effective and practical for the de-noising of PAM images.
A Noise-level-aware Framework for PET Image Denoising

Ye Li,Jianan Cui,Junyu Chen,Guodong Zeng,Scott Wollenweber,Floris Jansen,Se-In Jang,Kyungsang Kim,Kuang Gong,Quanzheng Li

DOI: https://doi.org/10.1007/978-3-031-17247-2_8

2022-03-16

Abstract:In PET, the amount of relative (signal-dependent) noise present in different body regions can be significantly different and is inherently related to the number of counts present in that region. The number of counts in a region depends, in principle and among other factors, on the total administered activity, scanner sensitivity, image acquisition duration, radiopharmaceutical tracer uptake in the region, and patient local body morphometry surrounding the region. In theory, less amount of denoising operations is needed to denoise a high-count (low relative noise) image than images a low-count (high relative noise) image, and vice versa. The current deep-learning-based methods for PET image denoising are predominantly trained on image appearance only and have no special treatment for images of different noise levels. Our hypothesis is that by explicitly providing the local relative noise level of the input image to a deep convolutional neural network (DCNN), the DCNN can outperform itself trained on image appearance only. To this end, we propose a noise-level-aware framework denoising framework that allows embedding of local noise level into a DCNN. The proposed is trained and tested on 30 and 15 patient PET images acquired on a GE Discovery MI PET/CT system. Our experiments showed that the increases in both PSNR and SSIM from our backbone network with relative noise level embedding (NLE) versus the same network without NLE were statistically significant with p<0.001, and the proposed method significantly outperformed a strong baseline method by a large margin.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning,Medical Physics

Noise-Resilient Ptychographic Imaging with Deep Learning

Single-frame Noisy Interferogram Phase Retrieval Using an End-to-end Deep Learning Network with Physical Information Constraints

Frequency-Relevant Residual Learning for Multi-Modal Image Denoising.

Ptychoformer: A Physics-Guided Deep Learning Framework for Ptychographic Imaging

Ptychographic phase retrieval via a deep‐learning‐assisted iterative algorithm

Noise-robust latent vector reconstruction in ptychography using deep generative models

An adaptive noise-blind-separation algorithm for ptychography

Unsupervised denoising of photoacoustic images based on the Noise2Noise network

Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising

Addressing image and Poisson noise deconvolution problem using deep learning approaches

Noise Reduction in Optical Coherence Tomography Images Using a Deep Neural Network with Perceptually-Sensitive Loss Function.

Remote Sensing Image Mixed Noise Denoising with Noise Parameter Estimation

3D-2D Neural Nets for Phase Retrieval in Noisy Interferometric Imaging

Background Denoising for Ptychography via Wigner Distribution Deconvolution

AI-enabled high-resolution scanning coherent diffraction imaging

Advanced Denoising for X-ray Ptychography

Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training

Deep learning optical image denoising research based on principal component estimation

Physics-based Noise Modeling for Extreme Low-light Photography

De-Noising of Photoacoustic Microscopy Images by Deep Learning

A Noise-level-aware Framework for PET Image Denoising