Volumetric Particle Imaging Using Differentiable Holography

Yang Wu,Jun Wang,Ni Chen

DOI: https://doi.org/10.1364/dh.2023.hth2c.2

2023-01-01

Abstract:Utilizing automatic differentiation and physical constraints, this paper presents an accurate forward model in digital holography, achieving superior volumetric particle field imaging compared to compressive holography without approximation assumptions.

What problem does this paper attempt to address?

Digital Holography of Particle Fields: Reconstruction by Use of Complex Amplitude.

G Pan,H Meng

DOI: https://doi.org/10.1364/ao.42.000827

IF: 1.9

2003-01-01

Applied Optics

Abstract:Digital holography appears to be a strong contender as the next-generation technology for holographic diagnostics of particle fields and holographic particle image velocimetry for flow field measurement. With the digital holographic approach, holograms are directly recorded by a digital camera and reconstructed numerically. This not only eliminates wet chemical processing and mechanical scanning, but also enables the use of complex amplitude information inaccessible by optical reconstruction, thereby allowing flexible reconstruction algorithms to achieve optimization of specific information. However, owing to the inherently low pixel resolution of solid-state imaging sensors, digital holography gives poor depth resolution for images, a problem that severely impairs the usefulness of digital holography especially in densely populated particle fields. This paper describes a technique that significantly improves particle axial-location accuracy by exploring the reconstructed complex amplitude information, compared with other numerical reconstruction schemes that merely mimic traditional optical reconstruction. This novel method allows accurate extraction of particle locations from forward-scattering particle holograms even at high particle loadings.
Multiplane Digital Holography Based on Extrapolation Iterations

Zhengzhong Huang,Cuifang Kuang,Lin Xu,Liangcai Cao

DOI: https://doi.org/10.1016/j.optcom.2020.126526

IF: 2.4

2020-01-01

Optics Communications

Abstract:The resolution of reconstructed images is significantly limited by the size of captured holograms in digital holography. A multiplane digital holographic algorithm based on extrapolation iterations is proposed herein. Holograms at different axial distances are captured with a shifting imaging sensor and then extrapolated to a larger hologram after padding. Multiple holograms provide multipoint constraints in diffraction calculation. Constraints based on the energy distribution and transmissions of an object are applied to extract the object function from the holograms. The method achieves a higher convergence speed for solving the inverse problem compared with the single-plane extrapolation method. We demonstrate that the reconstruction in multiple planes yields a better resolution than that in one plane.
Hybrid Digital Holographic Imaging System for Three-Dimensional Dense Particle Field Measurement

Lujie Cao,Gang Pan,Jeremy de Jong,Scott Woodward,Hui Meng

DOI: https://doi.org/10.1364/ao.47.004501

IF: 1.9

2008-01-01

Applied Optics

Abstract:To apply digital holography to the measurement of three-dimensional dense particle fields in large facilities, we have developed a hybrid digital holographic particle-imaging system. The technique combines the advantages of off-axis (side) scattering in suppressing speckle noise and on-axis (in-line) recording in lowering the digital sensor resolution requirement. A camera lens is attached to the digital sensor to compensate for the weak object wave from side scattering over a large recording distance. A simple numerical reconstruction algorithm is developed for holograms recorded with a lens without requiring complex and impractical mathematical corrections. We analyze the effect of image sensor resolution and off-axis angle on system performance and quantify the particle positioning accuracy of the system. The holographic system is successfully applied to the study of inertial particle clustering in isotropic turbulence.
Digital Holographic PIV for 3D Flow Measurement

Gang Pan,Hui Meng

DOI: https://doi.org/10.1115/imece2002-33173

2002-01-01

Abstract:We have developed a simple digital holographic PIV system for 3D measurement of turbulent and multiphase flow. This system records in-line holograms of tracer particles directly on a digital camera and reconstructs the 3D particle field numerically. Using a novel complex amplitude-based method, we are able to overcome the depth of focus problem and the speckle noise problem associated with the low resolution of digital sensors and the in-line holographic setup. In this paper we will give detailed description of our system, and present the preliminary result of the benchmark experiment using a water jet.
A Model Based Algorithm for Fast DPIV Computing

Chen Tian-ding,Li Hong-dong,Liu Ji-lin,Gu Wei-kang

DOI: https://doi.org/10.1631/bf02841175

2001-01-01

Journal of Zhejiang University SCIENCE A

Abstract:Traditional DPIV (Digital Particle Image Velocimetry) methods are mostly based on area-correlation (Willert, C. E., 1991). Though proven to be very time-consuming and very much error prone, they are widely adopted because of they are conceptually simple and easily implemented, and also because there are few alternatives. This paper proposes a non-correlation, conceptually new, fast and efficient approach for DPIV, which takes the nature of flow into consideration. An Incompressible Affined Flow Model (IAFM) is introduced to describe a flow that incorporates rational restraints into the computation. This IAFM, combined with a modified optical flow method-named Total Optical Flow Computation (TOFC), provides a linear system solution to DPIV. Experimental results on real images showed our method to be a very promising approach for DPIV.
Digital In-line Holographic PIV for 3 D Particulate Flow Diagnostics

G. Pan,H. Meng

2001-01-01

Abstract:We present a digital in-line holographic PIV system for diagnosing 3D particulate flows. This system records in-line holograms directly to a CCD camera and reconstructs them numerically. Since the pixel resolution of CCD cameras is much lower than that of holographic films, conventional particle extraction methods cannot yield high depth resolution that is important for velocity measurement and particulate flow diagnostics. To solve this problem, we have developed a novel particle extraction method based on the complex amplitude of reconstructed wave. With a simple in-line recording setup, this method can reach a depth resolution that is comparable with that of some off-axis holographic PIV systems. This method also alleviates the speckle noise problem intrinsic to in-line particle holography since speckles and particles can be clearly differentiated in complex wavefield. This technique is the basis for a digital in-line holographic PIV (HPIV) system with the addition of Concise Cross Correlation (CCC) and particle pairing algorithm (Pu and Meng 2000). Details of the system hardware and data processing method are described in this paper. The system was tested by both a simulated 3D flow around a collapsing spherical cavity and a laboratory particle suspension.
Recent Advances and Applications of Digital Holography in Multiphase Reactive/nonreactive Flows: a Review

Jianqing Huang,Weiwei Cai,Yingchun Wu,Xue-cheng Wu

DOI: https://doi.org/10.1088/1361-6501/ac32ea

IF: 2.398

2021-01-01

Measurement Science and Technology

Abstract:In various multiphase flows, the characterization of particle dynamics is important in the understanding of the interaction between particles and the surrounding flows. Digital holography (DH) is a versatile 3D imaging technique, which has shown great advantages in quantitative analysis and nonintrusive diagnosis of various particle fields. This review focuses on the advances and applications of DH in multiphase reactive/nonreactive flows in the last two decades. First, the basic principles of DH are introduced, including its mathematical background and representative experimental configurations. Then, the image processing algorithms for hologram reconstruction and automatic focusing are summarized, along with the methods for separating overlapping particles and tracking moving particles. As a prevailing and powerful tool, the recent applications of deep learning in processing holographic images is also included in this review. Furthermore, the applications of DH in the characterization of particle dynamics in multiphase reactive/nonreactive flows are surveyed in detail. Lastly, the review concludes with a discussion on the technical limits of DH and provides insights into its promising future research directions.
An Approach For Sensing Underwater Particles Using Digital Holographic Imaging

Chao Liu,Xiaoping Wang,Yingtian Hu,Ming Tang,Zheng Shi

DOI: https://doi.org/10.1117/12.2505602

2018-01-01

Abstract:The purpose of this paper is to use digital holographic imaging to capture the hologram of the underwater particles. Digital holography includes two steps: wavefront recording and wavefront reconstruction. In the experiments of the underwater particles, some technique issues, such as the principle of digital holography, zero-order item suppressing and reconstruction algorithm are discussed. The complex amplitude of the underwater particles is reconstructed using numerical methods based on the Kirchhoff-Fresnel integral formula. In this case, it enables reconstruction of the object wavefield in a plane at a distance d from hologram. This novel method for imaging the underwater particles using digital holography is an extremely valuable technique for the study of underwater particles fields as it has many advantages such as large depth of field, no focusing, high resolution and has the ability to record the wavefront information of all particles in the entire volume. Finally, this paper demonstrates it is possible for digital holographic imaging setup with less complexity to image underwater particles according to our laboratory results.
Adaptive In-Focus Particle Detection and Segmentation in Holographic 3d Image with Mechanism-Guided Machine Learning

Hang Zhang,Boyi Wang,Letian Zhang,Yue Zhao,Yu Wang,Jianhan Feng,Wei Xiao,Gaofeng Wang,Yingchun Wu,Xuecheng Wu

DOI: https://doi.org/10.1016/j.optlaseng.2024.108400

IF: 5.666

2024-01-01

Optics and Lasers in Engineering

Abstract:In the domain of three-dimensional (3D) particle holography, diverse imaging characteristics across particles of varying scales and positions pose a notable impediment to efficient information extraction. This research introduces an adaptive detection and segmentation approach employing mechanism-guided machine learning techniques. Not only the carefully chosen detection network but also the cropping pretreatment of input images are used to obtain the precise regions of interest (ROIs) of particles, inspired with the estimated particle size distribution under specific experimental conditions. Restricted by the blurred particle boundary and aiming for a robust method for hyperparameter, cluster-based segmentation is proposed with an extra feature map, which takes the unique change of object signals across different slices compared with background into account. Rigorous evaluation of different detection network configurations, training data, and segmentation methods are conducted on experimental data from an icing wind tunnel and calibration dot board. Concrete ablation experiments and analysis of the mechanism-guided operations (cropping and the introduction of the extra feature map) are provided. The influence of the hyperparameter in segmentation is also illustrated. To showcase its robustness, the proposed method is applied to segment a swirl spray particle field, demonstrating its versatility and applicability across varied particle field scenarios. The code and data for the paper are available at https://github.com/Adiazhang0426/HOLOAI/tree/master and https://drive.google.com/drive/folders/1C9QlaklXpApIpE8AukCNMTKUnZw_YcGJ?usp=drive_link.
Automatic Differentiation for All Photons Imaging to See Inside Volumetric Scattering Media

Tomohiro Maeda,Ankit Ranjan,Ramesh Raskar

DOI: https://doi.org/10.48550/arXiv.2006.01897

2020-06-03

Abstract:Imaging through dense scattering media - such as biological tissue, fog, and smoke - has applications in the medical and robotics fields. We propose a new framework using automatic differentiation for All Photons Imaging through homogeneous scattering media with unknown optical properties for non-invasive sensing and diagnostics. We overcome the need for the imaging target to be visible to the illumination source in All Photons Imaging, enabling practical and non-invasive imaging through turbid media with a simple optical setup. Our method does not require calibration to acquire the sensor position or optical properties of the media.

Image and Video Processing,Computer Vision and Pattern Recognition
Differentiable Imaging: A New Tool for Computational Optical Imaging

Ni Chen,Liangcai Cao,Ting-Chung Poon,Byoungho Lee,Edmund Y. Lam

DOI: https://doi.org/10.1002/apxr.202200118

2023-01-01

Abstract:The field of computational imaging has made significant advancements in recent years, yet it still faces limitations due to the restrictions imposed by traditional computational techniques. Differentiable programming offers a solution by combining the strengths of classical optimization and deep learning, enabling the creation of interpretable model‐based neural networks. Through the integration of physics into the modeling process, differentiable imaging, which employs differentiable programming in computational imaging, has the potential to overcome challenges posed by sparse, incomplete, and noisy data. As a result, it has the potential to play a key role in advancing the field of computational imaging and its various applications.
Generalizable Deep Learning Approach for 3D Particle Imaging using Holographic Microscopy

Shyam Kumar,Jiarong Hong

2024-11-25

Abstract:Despite its potential for label-free particle diagnostics, holographic microscopy is limited by specialized processing methods that struggle to generalize across diverse settings. We introduce a deep learning architecture leveraging human perception of longitudinal variation of diffracted patterns of particles, which enables highly generalizable analysis of 3D particle information with orders of magnitude improvement in processing speed. Trained with minimal synthetic and real holograms of simple particles, our method demonstrates exceptional performance on various challenging cases including those with high particle concentrations and noises and a wide range of particle sizes, complex shapes, and optical properties exceeding the diversity of the training datasets.

Optics
Differentiable Direct Volume Rendering

Sebastian Weiss,Rüdiger Westermann

DOI: https://doi.org/10.1109/TVCG.2021.3114769

2021-07-27

Abstract:We present a differentiable volume rendering solution that provides differentiability of all continuous parameters of the volume rendering process. This differentiable renderer is used to steer the parameters towards a setting with an optimal solution of a problem-specific objective function. We have tailored the approach to volume rendering by enforcing a constant memory footprint via analytic inversion of the blending functions. This makes it independent of the number of sampling steps through the volume and facilitates the consideration of small-scale changes. The approach forms the basis for automatic optimizations regarding external parameters of the rendering process and the volumetric density field itself. We demonstrate its use for automatic viewpoint selection using differentiable entropy as objective, and for optimizing a transfer function from rendered images of a given volume. Optimization of per-voxel densities is addressed in two different ways: First, we mimic inverse tomography and optimize a 3D density field from images using an absorption model. This simplification enables comparisons with algebraic reconstruction techniques and state-of-the-art differentiable path tracers. Second, we introduce a novel approach for tomographic reconstruction from images using an emission-absorption model with post-shading via an arbitrary transfer function.

Graphics
HoloDiffusion: Sparse Digital Holographic Reconstruction via Diffusion Modeling

Liu Zhang,Songyang Gao,Minghao Tong,Yicheng Huang,Zibang Zhang,Wenbo Wan,Qiegen Liu

DOI: https://doi.org/10.3390/photonics11040388

IF: 2.536

2024-04-22

Photonics

Abstract:In digital holography, reconstructed image quality can be primarily limited due to the inability of a single small aperture sensor to cover the entire field of a hologram. The use of multi-sensor arrays in synthetic aperture digital holographic imaging technology contributes to overcoming the limitations of sensor coverage by expanding the area for detection. However, imaging accuracy is affected by the gap size between sensors and the resolution of sensors, especially when dealing with a limited number of sensors. An image reconstruction method is proposed that combines physical constraint characteristics of the imaging object with a score-based diffusion model, aiming to enhance the imaging accuracy of digital holography technology with extremely sparse sensor arrays. Prior information of the sample is learned by the neural network in the diffusion model to obtain a score function, which alternately constrains the iterative reconstruction process with the underlying physical model. The results demonstrate that the structural similarity and peak signal-to-noise ratio of the reconstructed images using this method are higher than the traditional method, along with a strong generalization ability.

optics
Wavelet-based Depth-of-field Extension, Accurate Autofocusing, and Particle Pairing for Digital Inline Particle Holography.

Wu Yingchun,Wu Xuecheng,Yang Jing,Wang Zhihua,Gao Xiang,Zhou Binwu,Chen Linghong,Qiu Kunzan,Gerard Grehan,Cen Kefa

DOI: https://doi.org/10.1364/ao.53.000556

IF: 1.9

2014-01-01

Applied Optics

Abstract:Depth-of-field extension and accurate 3D position location are two important issues in digital holography for particle characterization and motion tracking. We propose a method of locating the axial positions of both opaque and transparent objects in the reconstructed 3D field in the wavelet domain. The spatial-frequency property of the reconstructed image is analyzed from the viewpoint of the point spread function of the digital inline holography. The reconstructed image is decomposed into high- and low-frequency subimages. By using the variance of the image gradient in the subimages as focus metrics, the depth-of-field of the synthesis image can be extended with all the particles focalized, and the focal plane of the object can be accurately determined. The method is validated by both simulated and experimental holograms of transparent spherical water droplets and opaque nonspherical coal particles. The extended-focus image is applied to the particle pairing in a digital holographic particle tracking velocimetry to obtain the 3D vector field.
Holographic particle image velocimetry: from film to digital recording

Hui Meng,Gang Pan,Ye Pu,Scott H Woodward

DOI: https://doi.org/10.1088/0957-0233/15/4/009

IF: 2.398

2004-03-19

Measurement Science and Technology

Abstract:Holographic particle image velocimetry (HPIV) offers potentially the best solution to volumetric measurements of the three-dimensional velocity fields in complex flows. However, the traditional film-based HPIV measurement is rather cumbersome, limiting its use to only a handful of groups worldwide. The newly emerged digital HPIV revolutionizes flow measurement science by providing a practical 3D velocimetry tool. It commands simple hardware that is similar to regular two-dimensional particle image velocimetry (PIV), yet it provides continuous (time-series) three-dimensional, three-component flow field data. Not only is the need for chemical processing eliminated, but also the cumbersome optical reconstruction is completely replaced by numerical reconstruction algorithms. Several breakthroughs have led to the development of the first practical and integrated digital HPIV systems. To explain the transition from film to digital recording, fundamental issues in HPIV are reviewed in this paper. Axial accuracy in HPIV measurement is ultimately limited by an inherent depth-of-focus problem, while information capacity is limited by inherent speckle noise. Information capacity is an important concept in HPIV, comprising the maximum acceptable seeding density multiplied by the sample volume depth along the optic axis. Both the axial accuracy and the information capacity are limited by the effective hologram aperture. The pursuit of a large hologram aperture in the past has resulted in further complexity in film-based HPIV systems. Digital HPIV, on the other hand, enjoys great simplicity of implementation and operation. A digital HPIV is also far more compact and rugged compared to existing film-based HPIV systems, making it suitable for duplication and commercialization. However, since digital sensors suffer from inferior pixel resolutions compared to films, the effective hologram aperture is much smaller in digital HPIV than that achievable in film-based HPIV. Alleviating this problem, digital HPIV also presents new possibilities in data processing such as the use of the complex amplitude of the reconstructed light wave to improve depth sensitivity and signal-to-noise ratio. Two examples of digital HPIV systems and measurement results are given. We believe digital HPIV can revitalize holographic particle imaging and bring it into the mainstream in much the same way that digital PIV brought PIV into widespread use a decade ago.

engineering, multidisciplinary,instruments & instrumentation
Dual-view Tomographic Particle Holography: Modeling and Light Scattering Effects

Yingchun Wu,Hongyu Zhang,Xuecheng Wu,Kefa Cen

DOI: https://doi.org/10.1016/j.jqsrt.2020.107232

2020-01-01

Abstract:Dual-view tomographic particle holography, combining tomography and holography, is a promising approach to yield high accurate 3D measurement of particle field. A standard model of particle hologram has been developed in the framework of generalized Lorenz-Mie theory to model and investigate the light scattering effects on three configurations of dual-view tomographic particle holography, that is, dual-beam dual-view Gabor inline holography, single beam dual-view combining Gabor inline holography with off-axis scattering holography, and single beam dual-view off-axis scattering holography. Dual-beam dual-view Gabor inline holography is dominated by the traditional Gabor inline hologram, but the two light scattering fields bring about small amplitude off-axis scattering inline recording hologram, negligible self-reference hologram and lensless Fourier hologram. The off-axis scattering inline recording hologram can produce additional glare points in the reconstructed particle image. The off-axis scattering holography reconstructs multiple glare light, and generates more than one intersections in the merged 3D reconstructed field in tomographic particle holography, leading to possible ghost particle image and implying new algorithm for particle detection to be developed. This model can provide standard hologram to processing algorithm development. (C) 2020 Elsevier Ltd. All rights reserved.
Machine learning holography for 3D particle field imaging

Siyao Shao,Kevin Mallery,S Santosh Kumar,Jiarong Hong,santosh kumar sankar

DOI: https://doi.org/10.1364/OE.379480

IF: 3.8

2020-01-23

Optics Express

Abstract:Siyao Shao, Kevin Mallery, S. Santosh Kumar, Jiarong Hong We propose a new learning-based approach for 3D particle field imaging using holography. Our approach uses a U-net architecture ... [Opt. Express 28, 2987-2999 (2020)]

optics
Progress on Lensless Digital Holography Imaging Based on Compressive Holographic Algorithm

Zhang Hua,Cao Liangcai,Jin Guofan,David Brady

DOI: https://doi.org/10.3788/lop57.080001

2020-01-01

Laser & Optoelectronics Progress

Abstract:Lensless digital holographic imaging could support high resolution, large field of view and three-dimensional (3D) imaging, but improving the resolution and quality of the reconstruction is still challenging. In this review paper, the compressive holographic models based on diffraction propagation method arc introduced. Compressive sensing arc developed based on total variation regularization and two-step iterative shrinkage/thresholding algorithm. The physical mechanism of removing two-order noise and twin image is discussed. A filter layer is designed to improve the signal to noise ratio of 3D reconstruction. A block-wise algorithm based on effective anti-aliasing region is proposed, which can improve computational efficiency of compressive holography. A single-shot compressive holographic model based on two-angle illumination beam is proposed. It effectively improves the axial resolution of 3D imaging. High-resolution 3D reconstruction of multilayer masks and particle flow field arc demonstrated by using this algorithm.
Path-space differentiable rendering of participating media

Cheng Zhang,Zihan Yu,Shuang Zhao

DOI: https://doi.org/10.1145/3450626.3459782

IF: 6.2

2021-08-31

ACM Transactions on Graphics

Abstract:Physics-based differentiable rendering---which focuses on estimating derivatives of radiometric detector responses with respect to arbitrary scene parameters---has a diverse array of applications from solving analysis-by-synthesis problems to training machine-learning pipelines incorporating forward-rendering processes. Unfortunately, existing general-purpose differentiable rendering techniques lack either the generality to handle volumetric light transport or the flexibility to devise Monte Carlo estimators capable of handling complex geometries and light transport effects. In this paper, we bridge this gap by showing how generalized path integrals can be differentiated with respect to arbitrary scene parameters. Specifically, we establish the mathematical formulation of generalized differential path integrals that capture both interfacial and volumetric light transport. Our formulation allows the development of advanced differentiable rendering algorithms capable of efficiently handling challenging geometric discontinuities and light transport phenomena such as volumetric caustics. We validate our method by comparing our derivative estimates to those generated using the finite differences. Further, to demonstrate the effectiveness of our technique, we compare both differentiable rendering and inverse rendering performance with state-of-the-art methods.

computer science, software engineering

Volumetric Particle Imaging Using Differentiable Holography

Digital Holography of Particle Fields: Reconstruction by Use of Complex Amplitude.

Multiplane Digital Holography Based on Extrapolation Iterations

Hybrid Digital Holographic Imaging System for Three-Dimensional Dense Particle Field Measurement

Digital Holographic PIV for 3D Flow Measurement

A Model Based Algorithm for Fast DPIV Computing

Digital In-line Holographic PIV for 3 D Particulate Flow Diagnostics

Recent Advances and Applications of Digital Holography in Multiphase Reactive/nonreactive Flows: a Review

An Approach For Sensing Underwater Particles Using Digital Holographic Imaging

Adaptive In-Focus Particle Detection and Segmentation in Holographic 3d Image with Mechanism-Guided Machine Learning

Automatic Differentiation for All Photons Imaging to See Inside Volumetric Scattering Media

Differentiable Imaging: A New Tool for Computational Optical Imaging

Generalizable Deep Learning Approach for 3D Particle Imaging using Holographic Microscopy

Differentiable Direct Volume Rendering

HoloDiffusion: Sparse Digital Holographic Reconstruction via Diffusion Modeling

Wavelet-based Depth-of-field Extension, Accurate Autofocusing, and Particle Pairing for Digital Inline Particle Holography.

Holographic particle image velocimetry: from film to digital recording

Dual-view Tomographic Particle Holography: Modeling and Light Scattering Effects

Machine learning holography for 3D particle field imaging

Progress on Lensless Digital Holography Imaging Based on Compressive Holographic Algorithm

Path-space differentiable rendering of participating media