Abstract:A densely-sampled light field (LF) is highly desirable in various applications, such as 3-D reconstruction, post-capture refocusing and virtual reality. However, it is costly to acquire such data. Although many computational methods have been proposed to reconstruct a densely-sampled LF from a sparsely-sampled one, they still suffer from either low reconstruction quality, low computational efficiency, or the restriction on the regularity of the sampling pattern. To this end, we propose a novel learning-based method, which accepts sparsely-sampled LFs with irregular structures, and produces densely-sampled LFs with arbitrary angular resolution accurately and efficiently. We also propose a simple yet effective method for optimizing the sampling pattern. Our proposed method, an end-to-end trainable network, reconstructs a densely-sampled LF in a coarse-to-fine manner. Specifically, the coarse sub-aperture image (SAI) synthesis module first explores the scene geometry from an unstructured sparsely-sampled LF and leverages it to independently synthesize novel SAIs, in which a confidence-based blending strategy is proposed to fuse the information from different input SAIs, giving an intermediate densely-sampled LF. Then, the efficient LF refinement module learns the angular relationship within the intermediate result to recover the LF parallax structure. Comprehensive experimental evaluations demonstrate the superiority of our method on both real-world and synthetic LF images when compared with state-of-the-art methods. In addition, we illustrate the benefits and advantages of the proposed approach when applied in various LF-based applications, including image-based rendering and depth estimation enhancement. The code is available at https://github.com/jingjin25/LFASR-FS-GAF.

Removing Foreground Occlusions in Light Field Using Micro-lens Dynamic Filter.

Unsupervised Dense Light Field Reconstruction with Occlusion Awareness

A Learning-Based Method Using Epipolar Geometry for Light Field Depth Estimation

DeOccNet: Learning to See Through Foreground Occlusions in Light Fields

Learning to restore light fields under low-light imaging

Light Field Reflection and Background Separation Network Based on Adaptive Focus Selection

Learning a Compact Spatial-Angular Representation for Light Field

Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses

Micro-Lens Image Stack Upsampling for Densely-Sampled Light Field Reconstruction

Light Field All-in-focus Image Fusion Based on Spatially-Guided Angular Information.

Light Field All-in-focus Image Fusion Based on Edge Enhanced Guided Filtering

Depth Estimation Algorithm for Light Field Data by Epipolar Image Analysis and Region Interpolation

Anti-Noise Light Field Depth Estimation Using Inline Occlusion Handling

Light Field Reconstruction using Efficient Pseudo 4D Epipolar-Aware Structure

Simulation Of Imaging And Refocusing For Light Field Photography

Multi-Stream Progressive Restoration for Low-Light Light Field Enhancement and Denoising

Occlusion-Model Guided Anti-Occlusion Depth Estimation in Light Field

Deep Coarse-to-Fine Dense Light Field Reconstruction With Flexible Sampling and Geometry-Aware Fusion

Automatic foreground segmentation using light field images

Light Field Image Compression with Sub-apertures Reordering and Adaptive Reconstruction.

Light Field Image Restoration via Latent Diffusion and Multi-View Attention