Abstract:Correlation Plenoptic Imaging (CPI) is a novel volumetric imaging technique that uses two sensors and the spatio-temporal correlations of light to detect both the spatial distribution and the direction of light. This novel approach to plenoptic imaging enables refocusing and 3D imaging with significant enhancement of both resolution and depth of field. However, CPI is generally slower than conventional approaches due to the need to acquire sufficient statistics for measuring correlations with an acceptable signal-to-noise ratio (SNR). We address this issue by implementing a Deep Learning application to improve image quality with undersampled frame statistics. We employ a set of experimental images reconstructed by a standard CPI architecture, at three different sampling ratios, and use it to feed a CNN model pre-trained through the transfer learning paradigm U-Net architecture with VGG-19 net for the encoding part. We find that our model reaches a Structural Similarity (SSIM) index value close to 1 both for the test sample (SSIM = [Formula: see text]) and in 5-fold cross validation (SSIM = [Formula: see text]); the results are also shown to outperform classic denoising methods, in particular for images with lower SNR. The proposed work represents the first application of Artificial Intelligence in the field of CPI and demonstrates its high potential: speeding-up the acquisition by a factor 20 over the fastest CPI so far demonstrated, enabling recording potentially 200 volumetric images per second. The presented results open the way to scanning-free real-time volumetric imaging at video rate, which is expected to achieve a substantial influence in various applications scenarios, from monitoring neuronal activity to machine vision and security.

Periodic patterns for resolution limit characterization of correlation plenoptic imaging

Effect of spatial spectrum overlap on fourier ptychographic microscopy

Plenoptic microscopy and photography from intensity correlations

Modulated Pattern Scanning Microscopy

Correlated-photon imaging at 10 volumetric images per second

Characterization of Defocused Coherent Imaging Systems with Periodic Objects

3D correlation imaging for localized phase disturbance mitigation

Innovative schemes for Correlation Plenoptic Imaging

Assessing the 3D resolution of refocused correlation plenoptic images using a general-purpose image quality estimator

Light-field imaging from position-momentum correlations

Correlation Hyperspectral Imaging

Ultra-High Spatial Resolutions in Photopatterning Molecular Orientations

Direct 3D imaging through spatial coherence of light

Resolution-Enhanced Ptychographic Modulation Imaging via Divergent Illumination

Super sub-wavelength patterns in photon coincidence detection

Spatial sub-Rayleigh imaging via structured speckle illumination

Compressive Direct Imaging of a Billion-Dimensional Optical Phase-Space

Improving the Resolution of Correlation Imaging via the Fluctuation Characteristics

Analytical form of the refocused images from correlation plenoptic imaging

Deep learning approach for denoising low-SNR correlation plenoptic images