Introduction to Plenoptic Imaging

Lu Fang

DOI: https://doi.org/10.1007/978-981-97-6915-5_1

2024-01-01

Abstract:AbstractPlenoptic is derived from the Latin words plenus (“full”) + optic and was proposed by Edward Adelson in 1991. The plenoptic function is a seven-dimensional function describing the three-dimensional viewing position, two-dimensional visual angle, one-dimensional wavelength, and one-dimensional time of a light ray in space.

What problem does this paper attempt to address?

Plenoptic Reconstruction

Lu Fang

DOI: https://doi.org/10.1007/978-981-97-6915-5_4

2024-01-01

Abstract:AbstractEmpowered by advanced plenoptic sensing systems, light-field imaging becomes one of the most extensively used methods for capturing 3D views of a scene. In contrast to the traditional input to a 3D graphics system, namely, scenes consisting of pre-defined geometric primitives with different materials and sets of lights, the input to a light field is only a set of 2D images which are informative and cost effective. Unfortunately, due to the limited sensor resolution, existing systems must balance the spatial and angular resolution, i.e., one can obtain dense sampling images in the spatial dimension but only sparse sampling images in the angular (viewing angle) dimension or vice versa.
High-Resolution Plenoptic Sensing

Lu Fang

DOI: https://doi.org/10.1007/978-981-97-6915-5_3

2024-01-01

Abstract:AbstractThe high-resolution plenoptic sensing system can capture light field with a high level of detail and clarity. Resolution refers to the amount of detail that can be discerned in the reconstructed image or light field. The cutting edge of high-resolution plenoptic sensing is gigapixel plenoptic sensing, specifically refers to a system capable of capturing images with billions of pixels, resulting in extremely high-resolution light field.
Plenoptic microscopy and photography from intensity correlations

Francesco V. Pepe,Francesco Di Lena,Augusto Garuccio,Davide Giannella,Alessandro Lupo,Gianlorenzo Massaro,Alessio Scagliola,Francesco Scattarella,Sergii Vasiukov,Milena D'Angelo

DOI: https://doi.org/10.1117/12.2594247

2024-09-14

Abstract:We present novel methods to perform plenoptic imaging at the diffraction limit by measuring intensity correlations of light. The first method is oriented towards plenoptic microscopy, a promising technique which allows refocusing and depth-of-field enhancement, in post-processing, as well as scanning free 3D imaging. To overcome the limitations of standard plenoptic microscopes, we propose an adaptation of Correlation Plenoptic Imaging (CPI) to the working conditions of microscopy. We consider and compare different architectures of CPI microscopes, and discuss the improved robustness with respect to previous protocols against turbulence around the sample. The second method is based on measuring correlations between the images of two reference planes, arbitrarily chosen within the tridimensional scene of interest, providing an unprecedented combination of image resolution and depth of field. The results lead the way towards the realization of compact designs for CPI devices.

Optics
Complete Plenoptic Imaging Using a Single Detector

Shuaishuai Zhu,Liang Gao,Yu Zhang,Jie Lin,Peng Jin

DOI: https://doi.org/10.1364/oe.26.026495

IF: 3.8

2018-01-01

Optics Express

Abstract:Multi-dimensional imaging is a powerful technique for many applications, such as biological analysis, remote sensing, and object recognition. Most existing multi-dimensional imaging systems rely on scanning or camera array, which make the system bulky and unstable. To some extent, these problems can be mitigated by employing compressed sensing algorithms. However, they are computationally expensive and highly rely on the ill-posed assumption that the information is sparse in a given domain. Here, we propose a snapshot spectral-volumetric imaging (SSVI) system by introducing the paradigm of light-field imaging into Fourier transform imaging spectroscopy. We demonstrate that SSVI can reconstruct a complete plenoptic function, P(x,y,z,θ,φ,λ,t), of the incoming light rays using a single detector. Compared with other multidimensional imagers, SSVI features prominent advantages in compactness, robustness, and low cost.
3D Plenoptic Representation of a Linear Scene

Cheng Bing,Li Xinxiao,Zheng Nanning,Quan Wei

2001-01-01

Abstract:This paper presents a novel 3D plenoptic function. We constrain camera motion to a line, and create a linear mosaic using a manifold mosaic. The plenoptic function is represented with three parameters: camera position along the axis, the angle between the ray and the centric axis, and the rotation angle in the vertical plane. Novel views are rendered by combining the appropriate captured rays in an efficient manner at the rendering time. Like panoramas, our method does not require recovery of geometric and photometric scene models. Moreover, it provides a much richer user experience by allowing the user to move freely in a linear region and observe significant parallax and lighting changes. Compared with either Lightfield or Lumigraph, it has a much smaller file size because a 3D only plenoptic function is constructed. Finally, an experiment with a synthetic environment is given to demonstrate its efficiency in capturing, construction and rendering of a linear scene.
Defocused Image Formation Model for Plenoptic Imaging

Yanqin Chen,Xin Jin,Qionghai Dai

DOI: https://doi.org/10.1364/3d.2018.jtu4a.40

2018-01-01

Abstract:This paper proposes to extend the PSF matrix of plenoptic imaging systems at focused depth to desirable defocused depth by multiplying a defocus approximation matrix. This extension is beneficial for saving time and hardware memory.
Toward Large-Scale Plenoptic Reconstruction

Lu Fang

DOI: https://doi.org/10.1007/978-981-97-6915-5_5

2024-01-01

Abstract:AbstractReconstructing real-world scenes with unparalleled levels of realism and detail has been a long-standing goal in the fields of computer vision and graphics. Achieving this goal necessitates coordinated efforts in both sensing techniques and plenoptic reconstruction algorithms.
Mind the Exit Pupil Gap: Revisiting the Intrinsics of a Standard Plenoptic Camera

Tim Michels,Daniel Mäckelmann,Reinhard Koch

DOI: https://doi.org/10.3390/s24082522

IF: 3.9

2024-04-16

Sensors

Abstract:Among the common applications of plenoptic cameras are depth reconstruction and post-shot refocusing. These require a calibration relating the camera-side light field to that of the scene. Numerous methods with this goal have been developed based on thin lens models for the plenoptic camera's main lens and microlenses. Our work addresses the often-overlooked role of the main lens exit pupil in these models, specifically in the decoding process of standard plenoptic camera (SPC) images. We formally deduce the connection between the refocusing distance and the resampling parameter for the decoded light field and provide an analysis of the errors that arise when the exit pupil is not considered. In addition, previous work is revisited with respect to the exit pupil's role, and all theoretical results are validated through a ray tracing-based simulation. With the public release of the evaluated SPC designs alongside our simulation and experimental data, we aim to contribute to a more accurate and nuanced understanding of plenoptic camera optics.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Optical center estimation for lenslet-based plenoptic cameras

Wei Hu,Mozhdeh Seifi,Erik Reinhard

DOI: https://doi.org/10.1109/MMSP.2017.8122214

2017-01-01

Abstract:Plenoptic cameras enable a variety of novel post-processing applications, including refocusing and single-shot 3D imaging. To achieve high accuracy, such applications typically require knowledge of intrinsic camera parameters. One such parameter is the location of the main lens' optical center relative to the sensor, which is required for modeling radially symmetric optical effects. We show that estimating this parameter can be achieved to an accuracy of less than half a pixel by utilising the symmetry inherent in each micro-image. Further, we show that estimating this parameter separately allows all other intrinsic camera parameters to be estimated with higher accuracy than can be achieved using a single optimization scheme, and leads to better vignetting correction than with an inaccurate optical center.
PlenoPatch: Patch-Based Plenoptic Image Manipulation.

Fang-Lue Zhang,Jue Wang,Eli Shechtman,Zi-Ye Zhou,Jiaxin Shi,Shi-Min Hu

DOI: https://doi.org/10.1109/TVCG.2016.2532329

2017-01-01

Abstract:Patch-based image synthesis methods have been successfully applied for various editing tasks on still images, videos and stereo pairs. In this work we extend patch-based synthesis to plenoptic images captured by consumer-level lenselet-based devices for interactive, efficient light field editing. In our method the light field is represented as a set of images captured from different viewpoints. We decompose the central view into different depth layers, and present it to the user for specifying the editing goals. Given an editing task, our method performs patch-based image synthesis on all affected layers of the central view, and then propagates the edits to all other views. Interaction is done through a conventional 2D image editing user interface that is familiar to novice users. Our method correctly handles object boundary occlusion with semi-transparency, thus can generate more realistic results than previous methods. We demonstrate compelling results on a wide range of applications such as hole-filling, object reshuffling and resizing, changing object depth, light field upscaling and parallax magnification.
Human Iris 3D Imaging using a micro-Plenoptic Camera

Hao Chen,Volker Sick,Maria A. Woodward,David Burke

DOI: https://doi.org/10.1364/boda.2017.bow3a.6

2017-01-01

Abstract:A micro-plenoptic system was developed to image the 3D structure of the human iris in single-exposure photographs. 3D iris patterns were effectively captured. Iris depth varied from 20~80 μm in a 4.9×3.9 mm2 area.
A New 3D Plenoptic Function-Linear Mosaic

张素,陈韩锋,徐乃平,刘允才

DOI: https://doi.org/10.3969/j.issn.1000-2758.2002.02.035

2002-01-01

Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University

Abstract:A 3D plenoptic function called linear mosaic (LM) is presented. LM is created by extracting appropriate circles from the forward images and putting them together in different blocks. It indexes rays according to three parameters: camera position along the track line, vertical angle and rotation angle. Because its representation is only a single frame of image, the LM is superior to other 3D plenoptic functions on compressing, storing and transmission. In the data structure of LM, pixels in a row correspond to a full-direction panorama, so the view can be produced by pre-loading adjacent rows for efficient rendering.
The Plenoptic Videos: Capturing, Rendering And Compression

Shing-chow Chan,To Ng King,Zhi-feng Gan,Kin-lok Chan,Heung-yeung Shum

DOI: https://doi.org/10.1109/ISCAS.2004.1328894

2004-01-01

Abstract:This paper presents a system for capturing and rendering a dynamic image-based representation called the plenoptic videos. It is a simplified version of light fields for dynamic environment, where user viewpoints are constrained along the camera plane of a linear array of video cameras. The system consists of a camera array of 8 Sony CCX-Z11 CCD cameras and eight Pentium 4 1.8 GHz computers connected together through a 100 BaseT LAN. Important issues such as multiple camera calibration, real-time compression, decompression and rendering are addressed. Experimental results demonstrated the usefulness of the proposed parallel processing based system in capturing and rendering high quality dynamic image-based representation using off-the-shelf equipment, and its potential applications in visualization and immersive television systems.
P‐7: A New Information Capacity Analysis of Integral Imaging

Jing Xie,Xu Liu,Haifeng Li,Caijie Van,Xinxing Xia

DOI: https://doi.org/10.1889/1.3499893

2010-01-01

Abstract:AbstractThree‐dimensional integral imaging is analyzed systematically, and a new analysis of the information capacity of integral imaging based on Geometrical‐Optics Phase Space theory is proposed. The calculation result provides an effective and comprehensive evaluation for integral imaging.
Tensor-based Plenoptic Image Denoising by Integrating Super-Resolution

Yun Liu,Na Qi,Zhiwei Xiong

DOI: https://doi.org/10.1016/j.image.2022.116789

IF: 3.453

2022-01-01

Signal Processing Image Communication

Abstract:In this paper, we propose a novel tensor-based denoising method targeting at plenoptic images which contain 4D light field (2D angular + 2D spatial) and 5D hyperspectral light field (2D angular + 2D spatial + 1D spectral). In order to make use of the high-dimension structural property of plenoptic images, we first generalize the intrinsic tensor sparsity measure to plenoptic images by extending the nonlocal similarity from the spatial dimension to the angular dimension. Second, to eliminate the sub-pixel misalignment of different views, we integrate the spatial super-resolution into denoising and exploit the spatial-angular correlation by utilizing the nonlocal similarity of the refined high-resolution central view. In the procedure of super-resolution, we utilize an intensity consistency criterion and a coordinate rationality criterion to facilitate the process of projection. The denoising performance can be boosted after back-projection performed on the refined high-resolution central view. Experimental results validate the superior performance of the proposed method on several plenoptic image datasets in terms of both subjective and objective quality.
Plenoptic sampling

Jin-Xiang Chai,Xin Tong,Shing-Chow Chan,Heung-Yeung Shum

DOI: https://doi.org/10.1145/344779.344932

2000-01-01

Abstract:This paper studies the problem of plenoptic sampling in image-based rendering (IBR). From a spectral analysis of light field signals and using the sampling theorem, we mathematically derive the analytical functions to determine the minimum sampling rate for light field rendering. The spectral support of a light field signal is bounded by the minimum and maximum depths only, no matter how complicated the spectral support might be because of depth variations in the scene. The minimum sampling rate for light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval. Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering.Plenoptic sampling goes beyond the minimum number of images needed for anti-aliased light field rendering. More significantly, it utilizes the scene depth information to determine the minimum sampling curve in the joint image and geometry space. The minimum sampling curve quantitatively describes the relationship among three key elements in IBR systems: scene complexity (geometrical and textural information), the number of image samples, and the output resolution. Therefore, plenoptic sampling bridges the gap between image-based rendering and traditional geometry-based rendering. Experimental results demonstrate the effectiveness of our approach.
All-In-Focus Polarimetric Imaging Based on an Integrated Plenoptic Camera with a Key Electrically Tunable LC Device

Mingce Chen,Zhexun Li,Mao Ye,Taige Liu,Chai Hu,Jiashuo Shi,Kewei Liu,Zhe Wang,Xinyu Zhang

DOI: https://doi.org/10.3390/mi13020192

IF: 3.4

2022-01-26

Micromachines

Abstract:In this paper, a prototyped plenoptic camera based on a key electrically tunable liquid-crystal (LC) device for all-in-focus polarimetric imaging is proposed. By using computer numerical control machining and 3D printing, the proposed imaging architecture can be integrated into a hand-held prototyped plenoptic camera so as to greatly improve the applicability for outdoor imaging measurements. Compared with previous square-period liquid-crystal microlens arrays (LCMLA), the utilized hexagonal-period LCMLA has remarkably increased the light utilization rate by ~15%. Experiments demonstrate that the proposed imaging approach can simultaneously realize both the plenoptic and polarimetric imaging without any macroscopic moving parts. With the depth-based rendering method, both the all-in-focus images and the all-in-focus degree of linear polarization (DoLP) images can be obtained efficiently. Due to the large depth-of-field advantage of plenoptic cameras, the proposed camera enables polarimetric imaging in a larger depth range than conventional 2D polarimetric cameras. Currently, the raw light field images with three polarization states including I0 and I60 and I120 can be captured by the proposed imaging architecture, with a switching time of several tens of milliseconds. Some local patterns which are selected as interested target features can be effectively suppressed or obviously enhanced by switching the polarization state mentioned. According to experiments, the visibility in scattering medium can also be apparently improved. It can be expected that the proposed polarimetric imaging approach will exhibit an excellent development potential.

chemistry, analytical,nanoscience & nanotechnology,instruments & instrumentation,physics, applied
Innovative schemes for Correlation Plenoptic Imaging

Gianlorenzo Massaro,Francesco Di Lena,Augusto Garuccio,Francesco V. Pepe,Milena D'Angelo

DOI: https://doi.org/10.1117/12.2568041

2024-09-14

Abstract:CPI is a novel imaging modality capable of addressing the intrinsic limitations of conventional plenoptic imaging - namely, the resolution loss and the sacrificed change of perspective, - while guaranteeing the typical advantages of plenotpic imaging: 3D imaging, refocusing of acquired pictures, in post-processing, and depth of field extension. In this work, we review a recently developed CPI scheme, named correlation plenoptic imaging between arbitrary planes, and derive the algorithm for image refocusing.

Optics
Depth Profile Measurement with Lenslet Images of the Plenoptic Camera

Peng Yang,Zhaomin Wang,Wei Zhang,Hongying Zhao,Weijuan Qu,Haimeng Zhao,Anand Asundi,Lei Yan

DOI: https://doi.org/10.1117/1.oe.57.3.033106

IF: 1.3

2018-01-01

Optical Engineering

Abstract:An approach for carrying out depth profile measurement of an object with the plenoptic camera is proposed. A single plenoptic image consists of multiple lenslet images. To begin with, these images are processed directly with a refocusing technique to obtain the depth map, which does not need to align and decode the plenoptic image. Then, a linear depth calibration is applied based on the optical structure of the plenoptic camera for depth profile reconstruction. One significant improvement of the proposed method concerns the resolution of the depth map. Unlike the traditional method, our resolution is not limited by the number of microlenses inside the camera, and the depth map can be globally optimized. We validated the method with experiments on depth map reconstruction, depth calibration, and depth profile measurement, with the results indicating that the proposed approach is both efficient and accurate. (C) 2018 Society of Photo-Optical Instrumentation Engineers (SPIE)

Introduction to Plenoptic Imaging

Plenoptic Reconstruction

High-Resolution Plenoptic Sensing

Plenoptic microscopy and photography from intensity correlations

Complete Plenoptic Imaging Using a Single Detector

3D Plenoptic Representation of a Linear Scene

Defocused Image Formation Model for Plenoptic Imaging

Toward Large-Scale Plenoptic Reconstruction

Mind the Exit Pupil Gap: Revisiting the Intrinsics of a Standard Plenoptic Camera

Optical center estimation for lenslet-based plenoptic cameras

PlenoPatch: Patch-Based Plenoptic Image Manipulation.

Human Iris 3D Imaging using a micro-Plenoptic Camera

A New 3D Plenoptic Function-Linear Mosaic

The Plenoptic Videos: Capturing, Rendering And Compression

P‐7: A New Information Capacity Analysis of Integral Imaging

Tensor-based Plenoptic Image Denoising by Integrating Super-Resolution

Plenoptic sampling

All-In-Focus Polarimetric Imaging Based on an Integrated Plenoptic Camera with a Key Electrically Tunable LC Device

Innovative schemes for Correlation Plenoptic Imaging

Depth Profile Measurement with Lenslet Images of the Plenoptic Camera