Analytical form of the refocused images from correlation plenoptic imaging

Gianlorenzo Massaro

DOI: https://doi.org/10.1364/oe.533669

IF: 3.8

2024-09-20

Optics Express

Abstract:Gianlorenzo Massaro Correlation plenoptic imaging (CPI) is emerging as a promising approach to light-field imaging (LFI), a technique for concurrently ... [Opt. Express 32, 35755-35770 (2024)]

optics

What problem does this paper attempt to address?

Assessing the 3D resolution of refocused correlation plenoptic images using a general-purpose image quality estimator

Gianlorenzo Massaro

2024-06-19

Abstract:Correlation plenoptic imaging (CPI) is emerging as a promising approach to light-field imaging (LFI), a technique enabling simultaneous measurement of light intensity distribution and propagation direction from a scene. LFI allows single-shot 3D sampling, offering fast 3D reconstruction for a wide range of applications. However, the array of micro-lenses typically used in LFI to obtain 3D information limits image resolution, which rapidly declines with enhanced volumetric reconstruction capabilities. CPI addresses this limitation by decoupling light-field information measurement using two photodetectors with spatial resolution, eliminating the need for micro-lenses. 3D information is encoded in a four-dimensional correlation function, which is decoded in post-processing to reconstruct images without the resolution loss seen in conventional LFI. This paper evaluates the tomographic performance of CPI, demonstrating that the refocusing reconstruction method provides axial sectioning capabilities comparable to conventional imaging systems. A general-purpose analytical approach based on image fidelity is proposed to quantitatively study axial and lateral resolution. This analysis fully characterizes the volumetric resolution of any CPI architecture, offering a comprehensive evaluation of its imaging performance.

Optics,Image and Video Processing
Innovative schemes for Correlation Plenoptic Imaging

Gianlorenzo Massaro,Francesco Di Lena,Augusto Garuccio,Francesco V. Pepe,Milena D'Angelo

DOI: https://doi.org/10.1117/12.2568041

2024-09-14

Abstract:CPI is a novel imaging modality capable of addressing the intrinsic limitations of conventional plenoptic imaging - namely, the resolution loss and the sacrificed change of perspective, - while guaranteeing the typical advantages of plenotpic imaging: 3D imaging, refocusing of acquired pictures, in post-processing, and depth of field extension. In this work, we review a recently developed CPI scheme, named correlation plenoptic imaging between arbitrary planes, and derive the algorithm for image refocusing.

Optics
Light-field imaging from position-momentum correlations

Davide Giannella,Gianlorenzo Massaro,Bohumil Stoklasa,Milena D'Angelo,Francesco V. Pepe

DOI: https://doi.org/10.1016/j.physleta.2023.129298

2024-01-29

Abstract:Correlation plenoptic imaging (CPI) is a light-field imaging technique employing intensity correlation measurements to simultaneously detect the spatial distribution and the propagation direction of light. Compared to standard methods, in which light-field images are directly encoded in intensity, CPI provides a significant enhancement of the volumetric reconstruction performance in terms of both achievable depth of field and 3D resolution. In this article, we present a novel CPI configuration where light-field information is encoded in correlations between position and momentum measurements, namely, points on a given object plane and points of the Fourier plane of the imaging lens. Besides the fundamental interest in retrieving the properties of position-momentum correlation, the proposed scheme overcomes practical limitations of previously proposed setups, providing higher axial homogeneity and robustness with respect to the identification of reference planes.

Optics
Plenoptic microscopy and photography from intensity correlations

Francesco V. Pepe,Francesco Di Lena,Augusto Garuccio,Davide Giannella,Alessandro Lupo,Gianlorenzo Massaro,Alessio Scagliola,Francesco Scattarella,Sergii Vasiukov,Milena D'Angelo

DOI: https://doi.org/10.1117/12.2594247

2024-09-14

Abstract:We present novel methods to perform plenoptic imaging at the diffraction limit by measuring intensity correlations of light. The first method is oriented towards plenoptic microscopy, a promising technique which allows refocusing and depth-of-field enhancement, in post-processing, as well as scanning free 3D imaging. To overcome the limitations of standard plenoptic microscopes, we propose an adaptation of Correlation Plenoptic Imaging (CPI) to the working conditions of microscopy. We consider and compare different architectures of CPI microscopes, and discuss the improved robustness with respect to previous protocols against turbulence around the sample. The second method is based on measuring correlations between the images of two reference planes, arbitrarily chosen within the tridimensional scene of interest, providing an unprecedented combination of image resolution and depth of field. The results lead the way towards the realization of compact designs for CPI devices.

Optics
Correlated-photon imaging at 10 volumetric images per second

Gianlorenzo Massaro,Paul Mos,Sergii Vasiukov,Francesco Di Lena,Francesco Scattarella,Francesco V. Pepe,Arin Ulku,Davide Giannella,Edoardo Charbon,Claudio Bruschini,Milena D'Angelo

DOI: https://doi.org/10.1038/s41598-023-39416-8

2023-08-08

Abstract:The correlation properties of light provide an outstanding tool to overcome the limitations of traditional imaging techniques. A relevant case is represented by correlation plenoptic imaging (CPI), a quantum-inspired volumetric imaging protocol employing spatio-temporally correlated photons from either entangled or chaotic sources to address the main limitations of conventional light-field imaging, namely, the poor spatial resolution and the reduced change of perspective for 3D imaging. However, the application potential of high-resolution imaging modalities relying on photon correlations is limited, in practice, by the need to collect a large number of frames. This creates a gap, unacceptable for many relevant tasks, between the time performance of correlated-light imaging and that of traditional imaging methods. In this article, we address this issue by exploiting the photon number correlations intrinsic in chaotic light, combined with a cutting-edge ultrafast sensor made of a large array of single-photon avalanche diodes (SPADs). This combination of source and sensor is embedded within a novel single-lens CPI scheme enabling to acquire 10 volumetric images per second. Our results place correlated-photon imaging at a competitive edge and prove its potential in practical applications.

Quantum Physics,Optics
Periodic patterns for resolution limit characterization of correlation plenoptic imaging

Francesco Scattarella,Gianlorenzo Massaro,Bohumil Stoklasa,Milena D'Angelo,Francesco V. Pepe

DOI: https://doi.org/10.1140/epjp/s13360-023-04322-5

2023-09-01

Abstract:The measurement of the spatio-temporal correlations of light provides an interesting tool to overcome the traditional limitations of standard imaging, such as the strong trade-off between spatial resolution and depth of field. In particular, using correlation plenoptic imaging, one can detect both the spatial distribution and the direction of light in a scene, pushing both resolution and depth of field to the fundamental limit imposed by wave-optics. This allows one to perform refocusing of different axial planes and three-dimensional reconstruction without any spatial scanning. In the present work, we investigate the resolution limit in a particular correlation plenoptic imaging scheme, by considering periodic test patterns, which provide, through analytical results, a deeper insight in the resolution properties of this second-order imaging technique, also in comparison with standard imaging.

Optics,Quantum Physics
3D correlation imaging for localized phase disturbance mitigation

Francesco V. Pepe,Milena D'Angelo

2024-06-15

Abstract:Correlation plenoptic imaging is a procedure to perform light-field imaging without spatial resolution loss, by measuring second-order spatio-temporal correlations of light. We investigate the possibility to use correlation plenoptic imaging to mitigate the effect of a phase disturbance in the propagation from the object to the main lens. We assume that this detrimental effect, that can be due to a turbulent medium, is localized at a specific distance from the lens, and is slowly varying in time. The mitigation of turbulence effects has already fostered the development of both light-field imaging and correlation imaging procedures. Here, we aim at merging these aspects, proposing a correlation light-field imaging method to overcome the effects of slowly varying turbulence, without the loss of lateral resolution, typical of traditional plenoptic imaging devices.

Optics,Quantum Physics
Deep learning approach for denoising low-SNR correlation plenoptic images

Francesco Scattarella,Domenico Diacono,Alfonso Monaco,Nicola Amoroso,Loredana Bellantuono,Gianlorenzo Massaro,Francesco V Pepe,Sabina Tangaro,Roberto Bellotti,Milena D'Angelo

DOI: https://doi.org/10.1038/s41598-023-46765-x

2023-11-10

Abstract:Correlation Plenoptic Imaging (CPI) is a novel volumetric imaging technique that uses two sensors and the spatio-temporal correlations of light to detect both the spatial distribution and the direction of light. This novel approach to plenoptic imaging enables refocusing and 3D imaging with significant enhancement of both resolution and depth of field. However, CPI is generally slower than conventional approaches due to the need to acquire sufficient statistics for measuring correlations with an acceptable signal-to-noise ratio (SNR). We address this issue by implementing a Deep Learning application to improve image quality with undersampled frame statistics. We employ a set of experimental images reconstructed by a standard CPI architecture, at three different sampling ratios, and use it to feed a CNN model pre-trained through the transfer learning paradigm U-Net architecture with VGG-19 net for the encoding part. We find that our model reaches a Structural Similarity (SSIM) index value close to 1 both for the test sample (SSIM = [Formula: see text]) and in 5-fold cross validation (SSIM = [Formula: see text]); the results are also shown to outperform classic denoising methods, in particular for images with lower SNR. The proposed work represents the first application of Artificial Intelligence in the field of CPI and demonstrates its high potential: speeding-up the acquisition by a factor 20 over the fastest CPI so far demonstrated, enabling recording potentially 200 volumetric images per second. The presented results open the way to scanning-free real-time volumetric imaging at video rate, which is expected to achieve a substantial influence in various applications scenarios, from monitoring neuronal activity to machine vision and security.
Lensless computationally defined confocal incoherent imaging with a Fresnel zone plane as coded aperture

Jihui Chen,Feng Wang,Yulong Li,Xing Zhang,Ke Yao,Zanyang Guan,Xiangming Liu

DOI: https://doi.org/10.1364/ol.497086

IF: 3.6

2023-08-22

Optics Letters

Abstract:Jihui Chen, Feng Wang, Yulong Li, Xing Zhang, Ke Yao, Zanyang Guan, Xiangming Liu We present a Fresnel zone plate (FZP) mask-based system for single-shot lensless confocal imaging. The system uses an FZP as coded ... [Opt. Lett. 48, 4520-4523 (2023)]

optics
Interferometric Lensless Imaging: Rank-One Projections of Image Frequencies With Speckle Illuminations

Olivier Leblanc,Matthias Hofer,Siddharth Sivankutty,Hervé Rigneault,Laurent Jacques

DOI: https://doi.org/10.1109/tci.2024.3359178

IF: 5.4

2024-02-16

IEEE Transactions on Computational Imaging

Abstract:Lenslessillumination single-pixel imaging with a multicore fiber (MCF) is a computational imaging technique that enables potential endoscopic observations of biological samples at cellular scale. In this work, we show that this technique is tantamount to collecting multiple symmetric rank-one projections (SROP) of an interferometric matrix—a matrix encoding the spectral content of the sample image. In this model, each SROP is induced by the complex sketching vector shaping the incident light wavefront with a spatial light modulator (SLM), while the projected interferometric matrix collects up to image frequencies for a -core MCF. While this scheme subsumes previous sensing modalities, such as raster scanning (RS) imaging with beamformed illumination, we demonstrate that collecting the measurements of random SLM configurations—and thus acquiring SROPs—allows us to estimate an image of interest if and scale log-linearly with the image sparsity level This demonstration is achieved both theoretically, with a specific restricted isometry analysis of the sensing scheme, and with extensive Monte Carlo experiments. On a practical side, we perform a single calibration of the sensing system robust to certain deviations to the theoretical model and independent of the sketching vectors used during the imaging phase. Experimental results made on an actual MCF system demonstrate the effectiveness of this imaging procedure on a benchmark image.

engineering, electrical & electronic,imaging science & photographic technology
Computational refocusing in phase-resolved confocal microscopy

Martin Schnell,Melanie King,Sam Buercklin,Paulo Sarriugarte,Rainer Hillenbrand,P. Scott Carney

DOI: https://doi.org/10.1364/OL.496111

2024-11-12

Abstract:We demonstrate numerical refocusing in coherent confocal laser scanning microscopy based on synthetic optical holography. This physics-based approach implements a computational propagation on the complex signal recovered in synthetic holography consistent with the wave physics and the parameters of the microscope. An experimental demonstration is shown to restore an in-focus image of a test object from data acquired at several focal plane off-sets. Numerical refocusing can provide focused views on samples with large height variation with a potential application in confocal optical surface profiling.

Optics
Correlation Hyperspectral Imaging

Gianlorenzo Massaro,Francesco V. Pepe,and Milena D'Angelo

DOI: https://doi.org/10.1103/physrevlett.133.183802

IF: 8.6

2024-10-29

Physical Review Letters

Abstract:Hyperspectral imaging aims at providing information on both the spatial and the spectral distribution of light, with high resolution. However, state-of-the-art protocols are characterized by an intrinsic trade-off imposing to sacrifice either resolution or image acquisition speed. We address this li... [Phys. Rev. Lett. 133, 183802] Published Mon Oct 28, 2024

physics, multidisciplinary
Freeform Liquid–Crystal Polarization Imaging Optics: A Practical Approach from Design to Fabrication

Chunyang Pei,Jiacheng Weng,Tian Shu,Linyue Fang,Guangyin Hu,Yingli Liu,Ting Chen,Rengmao Wu,Haifeng Li,Xu Liu

DOI: https://doi.org/10.1002/lpor.202400204

2024-01-01

LASER & PHOTONICS REVIEWS

Abstract:Freeform liquid-crystal polarization imaging optics (LPIOs) have vast potential for application in next-generation interactive displays like augmented reality and virtual reality. However, tailoring the phase profile of a freeform LPIO for flexible aberration correction and diffraction efficiency control is still a big challenge due to the difficulties arising from the non-zero etendue of an imaging system and the finite bandwidth of LPIOs. Here, a practical approach to creating freeform LPIOs from design to fabrication is proposed. A freeform polarization exposure is developed to produce a flexible, precise, and ultra-high-resolution photoalignment pattern for a freeform LPIO. A cooperative optimization is performed between the freeform LPIO and the freeform polarization exposure system to find the optimal phase of the freeform LPIO and the optimal surface profiles of the freeform lenses employed in the exposure system to produce freeform wavefronts. The optimal phase of the freeform LPIO is then achieved by superposition interference of the engineered freeform wavefronts via freeform polarization exposure. The effectiveness of the approach and the advantages of freeform LPIOs is experimentally demonstrated. The proposed approach will significantly promote the widespread applications of freeform LPIOs in the field of optics and photonics. A practical approach to creating freeform LPIOs with flexible aberration correction and diffraction efficiency control from design to fabrication is proposed. A freeform polarization exposure is developed to produce a flexible, precise, and ultra-high-resolution photoalignment pattern for a freeform LPIO. The proposed approach will significantly promote the widespread applications of freeform LPIOs in the field of optics and photonics. image
All-In-Focus Polarimetric Imaging Based on an Integrated Plenoptic Camera with a Key Electrically Tunable LC Device

Mingce Chen,Zhexun Li,Mao Ye,Taige Liu,Chai Hu,Jiashuo Shi,Kewei Liu,Zhe Wang,Xinyu Zhang

DOI: https://doi.org/10.3390/mi13020192

IF: 3.4

2022-01-26

Micromachines

Abstract:In this paper, a prototyped plenoptic camera based on a key electrically tunable liquid-crystal (LC) device for all-in-focus polarimetric imaging is proposed. By using computer numerical control machining and 3D printing, the proposed imaging architecture can be integrated into a hand-held prototyped plenoptic camera so as to greatly improve the applicability for outdoor imaging measurements. Compared with previous square-period liquid-crystal microlens arrays (LCMLA), the utilized hexagonal-period LCMLA has remarkably increased the light utilization rate by ~15%. Experiments demonstrate that the proposed imaging approach can simultaneously realize both the plenoptic and polarimetric imaging without any macroscopic moving parts. With the depth-based rendering method, both the all-in-focus images and the all-in-focus degree of linear polarization (DoLP) images can be obtained efficiently. Due to the large depth-of-field advantage of plenoptic cameras, the proposed camera enables polarimetric imaging in a larger depth range than conventional 2D polarimetric cameras. Currently, the raw light field images with three polarization states including I0 and I60 and I120 can be captured by the proposed imaging architecture, with a switching time of several tens of milliseconds. Some local patterns which are selected as interested target features can be effectively suppressed or obviously enhanced by switching the polarization state mentioned. According to experiments, the visibility in scattering medium can also be apparently improved. It can be expected that the proposed polarimetric imaging approach will exhibit an excellent development potential.

chemistry, analytical,nanoscience & nanotechnology,instruments & instrumentation,physics, applied
Characterization of Defocused Coherent Imaging Systems with Periodic Objects

Gianlorenzo Massaro,Milena D'Angelo

DOI: https://doi.org/10.3390/s24216885

IF: 3.9

2024-10-26

Sensors

Abstract:Recent advancements in quantum and quantum-inspired imaging techniques have enabled high-resolution 3D imaging through photon correlations. These techniques exhibit reduced degradation of image resolution for out-of-focus samples compared to conventional methods (i.e., intensity-based incoherent imaging). A key advantage of these correlation-based approaches is their independence from the system numerical aperture (NA). Interestingly, both improved resolution of defocused images and NA-independent scaling are linked to the spatial coherence of light. This suggests that while correlation measurements exploit spatial coherence, they are not essential for achieving this imaging advantage. This discovery has led to the development of optical systems that achieve similar performance by using spatially coherent illumination and relying on intensity measurements: direct 3D imaging with NA-independent resolution was recently demonstrated in a correlation-free setup using LED light. Here, we explore the physics behind the enhanced performance of defocused coherent imaging, showing that it arises from the modification of the sample's spatial harmonic content due to diffraction, unlike the blurring seen in conventional imaging. The results we present are crucial for understanding the implications of the physical differences between coherent and incoherent imaging, and are expected to pave the way for the practical application of the discovered phenomena.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Super-resolved Refocusing with a Plenoptic Camera

Zhiliang Zhou,Yan Yuan,Bin Xiangli,Lulu Qian

DOI: https://doi.org/10.1117/12.872447

2011-01-01

Abstract:This paper presents an approach to enhance the resolution of refocused images by super resolution methods. In plenoptic imaging, we demonstrate that the raw sensor image can be divided to a number of low-resolution angular images with sub-pixel shifts between each other. The sub-pixel shift, which defines the super-resolving ability, is mathematically derived by considering the plenoptic camera as equivalent camera arrays. We implement simulation to demonstrate the imaging process of a plenoptic camera. A high-resolution image is then reconstructed using maximum a posteriori (MAP) super resolution algorithms. Without other degradation effects in simulation, the super resolved image achieves a resolution as high as predicted by the proposed model. We also build an experimental setup to acquire light fields. With traditional refocusing methods, the image is rendered at a rather low resolution. In contrast, we implement the super-resolved refocusing methods and recover an image with more spatial details. To evaluate the performance of the proposed method, we finally compare the reconstructed images using image quality metrics like peak signal to noise ratio (PSNR).
High-resolution, large field-of-view label-free imaging via aberration-corrected, closed-form complex field reconstruction

Ruizhi Cao,Cheng Shen,Changhuei Yang

2023-09-02

Abstract:Computational imaging methods empower modern microscopy with the ability of producing high-resolution, large field-of-view, aberration-free images. One of the dominant computational label-free imaging methods, Fourier ptychographic microscopy (FPM), effectively increases the spatial-bandwidth product of conventional microscopy by using multiple tilted illuminations to achieve high-throughput imaging. However, its iterative reconstruction method is prone to parameter selection, can be computationally expensive and tends to fail under excessive aberrations. Recently, spatial Kramers-Kronig methods show it is possible to analytically reconstruct complex field but lacks the ability of correcting aberrations or providing extended resolution enhancement. Here, we present a closed-form method, termed APIC, which weds the strengths of both methods. A new analytical phase retrieval framework is established in APIC, which demonstrates, for the first time, the feasibility of analytically reconstructing the complex field associated with darkfield measurements. In addition, APIC can analytically retrieve complex aberrations of an imaging system with no additional hardware. By avoiding iterative algorithms, APIC requires no human designed convergence metric and always obtains a closed-form complex field solution. The faithfulness and correctness of APIC's reconstruction are guaranteed due to its analytical nature. We experimentally demonstrate that APIC gives correct reconstruction result while FPM fails to do so when constrained to the same number of measurements. Meanwhile, APIC achieves 2.8 times faster computation using image tile size of 256 (length-wise). We also demonstrate APIC is unprecedentedly robust against aberrations compared to FPM - APIC is capable of addressing aberration whose maximal phase difference exceeds 3.8${\pi}$ when using a NA 0.25 objective in experiment.

Optics,Image and Video Processing
Defocused Image Formation Model for Plenoptic Imaging

Yanqin Chen,Xin Jin,Qionghai Dai

DOI: https://doi.org/10.1364/3d.2018.jtu4a.40

2018-01-01

Abstract:This paper proposes to extend the PSF matrix of plenoptic imaging systems at focused depth to desirable defocused depth by multiplying a defocus approximation matrix. This extension is beneficial for saving time and hardware memory.
Interferometric single-pixel imaging with a multicore fiber

Olivier Leblanc,Matthias Hofer,Siddharth Sivankutty,Hervé Rigneault,Laurent Jacques

2023-07-17

Abstract:Lensless illumination single-pixel imaging with a multicore fiber (MCF) is a computational imaging technique that enables potential endoscopic observations of biological samples at cellular scale. In this work, we show that this technique is tantamount to collecting multiple symmetric rank-one projections (SROP) of a Hermitian \emph{interferometric} matrix -- a matrix encoding the spectral content of the sample image. In this model, each SROP is induced by the complex \emph{sketching} vector shaping the incident light wavefront with a spatial light modulator (SLM), while the projected interferometric matrix collects up to $O(Q^2)$ image frequencies for a $Q$-core MCF. While this scheme subsumes previous sensing modalities, such as raster scanning (RS) imaging with beamformed illumination, we demonstrate that collecting the measurements of $M$ random SLM configurations -- and thus acquiring $M$ SROPs -- allows us to estimate an image of interest if $M$ and $Q$ scale linearly (up to log factors) with the image sparsity level, hence requiring much fewer observations than RS imaging or a complete Nyquist sampling of the $Q \times Q$ interferometric matrix. This demonstration is achieved both theoretically, with a specific restricted isometry analysis of the sensing scheme, and with extensive Monte Carlo experiments. Experimental results made on an actual MCF system finally demonstrate the effectiveness of this imaging procedure on a benchmark image.

Signal Processing

Analytical form of the refocused images from correlation plenoptic imaging

Assessing the 3D resolution of refocused correlation plenoptic images using a general-purpose image quality estimator

Innovative schemes for Correlation Plenoptic Imaging

Light-field imaging from position-momentum correlations

Plenoptic microscopy and photography from intensity correlations

Correlated-photon imaging at 10 volumetric images per second

Periodic patterns for resolution limit characterization of correlation plenoptic imaging

3D correlation imaging for localized phase disturbance mitigation

Deep learning approach for denoising low-SNR correlation plenoptic images

Lensless computationally defined confocal incoherent imaging with a Fresnel zone plane as coded aperture

Interferometric Lensless Imaging: Rank-One Projections of Image Frequencies With Speckle Illuminations

Computational refocusing in phase-resolved confocal microscopy

Correlation Hyperspectral Imaging

Freeform Liquid–Crystal Polarization Imaging Optics: A Practical Approach from Design to Fabrication

All-In-Focus Polarimetric Imaging Based on an Integrated Plenoptic Camera with a Key Electrically Tunable LC Device

Characterization of Defocused Coherent Imaging Systems with Periodic Objects

Super-resolved Refocusing with a Plenoptic Camera

High-resolution, large field-of-view label-free imaging via aberration-corrected, closed-form complex field reconstruction

Defocused Image Formation Model for Plenoptic Imaging

Interferometric single-pixel imaging with a multicore fiber