Abstract:Light field imaging can encode abundant scene information, including the intensity and direction of light rays, into 4D light field images. However, the limited number of sensors in commercial light field cameras leads to a trade-off between spatial and angular resolutions. In this paper, an angular super-resolution framework is proposed to synthesize new views and overcome hardware restrictions. First, light field intrinsic feature convolution is proposed to extract intrinsic information, i.e., scene content, complete view correlations, and epipolar structures. Consequently, spatial, angular, and cross-domain information can be preserved in the extracted features. Second, the spatial-angular and depth streams are proposed based on the light field intrinsic feature convolution to synthesize high angular resolution light fields. The spatial-angular stream utilizes the light field intrinsic information to improve the angular resolution, whereas the depth stream disentangles the geometric information from the extracted light field intrinsic features, which is used to warp the given sub-aperture images to the new view positions. Both streams can synthesize high-quality intermediate results, where the intrinsic and geometric information are utilized separately. Finally, a confidence-based stream fusion module is proposed to fuse the outputs from the two streams, achieving the joint employment of light field intrinsic and geometric information and solving the problem of insufficient information exploration in the current methods. We conduct a series of experiments to validate the effectiveness of each component in the framework and demonstrate that our method can achieve state-of-the-art performance in various scenes.(c) 2023 Elsevier B.V. All rights reserved.

All-in-depth Via Cross-baseline Light Field Camera.

A Learning-Based Method Using Epipolar Geometry for Light Field Depth Estimation

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

Three Dimensional Reconstruction Using a Lenslet Light Field Camera

Depth Estimation Algorithm for Light Field Data by Epipolar Image Analysis and Region Interpolation

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Light-field-depth-estimation Network Based on Epipolar Geometry and Image Segmentation.

Depth Fused from Intensity Range and Blur Estimation for Light-Field Cameras

Depth Extraction of Low-Texture Region Using A Light Field Camera

Light Field Depth Estimation Exploiting Linear Structure in EPI

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

LF-fusion: Dense and Accurate 3D Reconstruction from Light Field Images

Light Field Depth Estimation via Epipolar Plane Image Analysis and Locally Linear Embedding

Depth Estimation for Light-Field Images Using Stereo Matching and Convolutional Neural Networks

High quality depth maps from stereo matching and ToF camera.

Omnidirectional Dense SLAM for Back-to-back Fisheye Cameras

Light field angular super-resolution based on intrinsic and geometric information.

Depth Estimation by Analyzing Intensity Distribution for Light-Field Cameras.

How Depth Estimation in Light Fields Can Benefit from Super-Resolution?