Abstract:Plenoptic cameras have been increasingly applied in three-dimensional (3D) optical imaging and measurements. Micro-lens array is the key component to capture the images at different depths simultaneously to realize the 3D reconstruction of the scenery. However, only a part of the captured information will be used eventually when focusing on specific objects, which implies a waste of time and resources that are incompetent for fast-response tasks like gesture recognition. Furthermore, it is still quite challenging to accurately calibrate the depths under complex circumstances, even sometimes losing a part of the target information due to the low signal-to-noise ratio. In this paper, a new hybrid optical system is developed combining a geometrical waveguide and micro-lens array to project a virtual scale network for quantitative depth calibration and fast-tracking of targets. Such a methodology based on the augmented reality (AR) mechanism helps to rapidly scope the targeted objects/features without reconstructing the full range model, significantly saving the processing time for a low-latency response. By establishing a geometrical model to quantitatively correlate the images in the auxiliary coordinate system and the virtual scale coordinate system, the depth error caused by the plenoptic camera can be calibrated and corrected. The coefficient of determination R 2 is used to evaluate the depth accuracy of 3D images and acts as the threshold to control the depth correction iterations. The closer the value of R 2 is to 1, the more accurate the depth information is. Experiments proved that even under complex backgrounds or insufficient light, in virtue of the virtual coordinate networks, backgrounds could be rapidly filtered, and targets were effectively identified with accurate depth information. The algorithm flow chart for the correction of depth error is given. In such a way, the system can achieve faster and more accurate capturing of 3D objects in a real-time manner than conventional plenoptic cameras.

High-Accuracy Depth Estimation of Nematic Liquid Crystal Microlens Arrays Based on Convex Optimization Theory

A Learning-Based Method Using Epipolar Geometry for Light Field Depth Estimation

Three Dimensional Reconstruction Using a Lenslet Light Field Camera

Calibration-free Deep Optics for Depth Estimation with Precise Simulation

Machine learning guided rapid focusing with sensor-less aberration corrections.

High-resolution light field imaging based on liquid crytal microlens arrays with ZnO microstructure orientation

Chromatically Multi-Focal Optics Based on Micro-Lens Array Design.

Light-field-depth-estimation Network Based on Epipolar Geometry and Image Segmentation.

Depth map enhancement based on color and depth consistency

Multi-channel Residual Network Model for Accurate Estimation of Spatially-Varying and Depth-Dependent Defocus Kernels.

10‐2: A Deep Depth of Field Near Eye Light Field Displays Utilizing LC Lens and Dual‐layer LCDs

Depth-of-Field-Extended Plenoptic Camera Based on Tunable Multi-Focus Liquid-Crystal Microlens Array

Liquid-crystal-display-based touchable light field three-dimensional display using display-capture mapping calibration.

Depth Estimation Algorithm for Light Field Data by Epipolar Image Analysis and Region Interpolation

Defocusing Response Function Optimization in Depth Estimation of Boundary Complex Light Field Image

Depth error correction for plenoptic cameras based on an innovative AR system combining geometrical waveguide and micro-lens array

Self-assembled microlens array with controllable curvatures for integral imaging 3D display

Aberration-robust monocular passive depth sensing using a meta-imaging camera

Resolution Improvement of Light Field Imaging via a Nematic Liquid Crystal Microlens with Added Multi-Walled Carbon Nanotubes

Depth Fused from Intensity Range and Blur Estimation for Light-Field Cameras

LC-based lightfield camera prototype for rapidly creating target images optimized by finely adjusting several key coefficients and a LC-guided refocusing-rendering