Abstract:Light field imaging can encode abundant scene information, including the intensity and direction of light rays, into 4D light field images. However, the limited number of sensors in commercial light field cameras leads to a trade-off between spatial and angular resolutions. In this paper, an angular super-resolution framework is proposed to synthesize new views and overcome hardware restrictions. First, light field intrinsic feature convolution is proposed to extract intrinsic information, i.e., scene content, complete view correlations, and epipolar structures. Consequently, spatial, angular, and cross-domain information can be preserved in the extracted features. Second, the spatial-angular and depth streams are proposed based on the light field intrinsic feature convolution to synthesize high angular resolution light fields. The spatial-angular stream utilizes the light field intrinsic information to improve the angular resolution, whereas the depth stream disentangles the geometric information from the extracted light field intrinsic features, which is used to warp the given sub-aperture images to the new view positions. Both streams can synthesize high-quality intermediate results, where the intrinsic and geometric information are utilized separately. Finally, a confidence-based stream fusion module is proposed to fuse the outputs from the two streams, achieving the joint employment of light field intrinsic and geometric information and solving the problem of insufficient information exploration in the current methods. We conduct a series of experiments to validate the effectiveness of each component in the framework and demonstrate that our method can achieve state-of-the-art performance in various scenes.(c) 2023 Elsevier B.V. All rights reserved.

Light Field Image Super-Resolution Based on Raw Data with Transformers

Light field angular super-resolution based on structure and scene information

Light field angular super-resolution based on intrinsic and geometric information.

Light Field Image Super-Resolution Network Based on Angular Difference Enhancement

Light Field Image Super-Resolution Using Deformable Convolution

Light Field Spatial Super-resolution Via CNN Guided by A Single High-resolution RGB Image.

Detail-Preserving Transformer for Light Field Image Super-Resolution

Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization

Light field super-resolution using complementary-view feature attention

View Position Prior-Supervised Light Field Angular Super-Resolution Network with Asymmetric Feature Extraction and Spatial-Angular Interaction.

Light Field Spatial Super-Resolution Via Geometric Feature Interaction

Light Field Spatial Super-Resolution Using Deep Efficient Spatial-Angular Separable Convolution.

Light Field Image Super-Resolution Based on Dual Learning and Deep Fourier Channel Attention

Research on super-resolution reconstruction of light field images based on spatial angle information decoupling

Light‐field image super‐resolution with depth feature by multiple‐decouple and fusion module

Boosting Light Field Spatial Super-Resolution via Masked Light Field Modeling

Light Field Angular Super-Resolution Network Based on Convolutional Transformer and Deep Deblurring

Light field image super-resolution using a Content-Aware Spatial-Angular Interaction network

Physics-Informed Ensemble Representation for Light-Field Image Super-Resolution

LGFN: Lightweight Light Field Image Super-Resolution using Local Convolution Modulation and Global Attention Feature Extraction

End-to-End Light Field Spatial Super-Resolution Network Using Multiple Epipolar Geometry