Abstract:A densely-sampled light field (LF) is highly desirable in various applications, such as 3-D reconstruction, post-capture refocusing and virtual reality. However, it is costly to acquire such data. Although many computational methods have been proposed to reconstruct a densely-sampled LF from a sparsely-sampled one, they still suffer from either low reconstruction quality, low computational efficiency, or the restriction on the regularity of the sampling pattern. To this end, we propose a novel learning-based method, which accepts sparsely-sampled LFs with irregular structures, and produces densely-sampled LFs with arbitrary angular resolution accurately and efficiently. We also propose a simple yet effective method for optimizing the sampling pattern. Our proposed method, an end-to-end trainable network, reconstructs a densely-sampled LF in a coarse-to-fine manner. Specifically, the coarse sub-aperture image (SAI) synthesis module first explores the scene geometry from an unstructured sparsely-sampled LF and leverages it to independently synthesize novel SAIs, in which a confidence-based blending strategy is proposed to fuse the information from different input SAIs, giving an intermediate densely-sampled LF. Then, the efficient LF refinement module learns the angular relationship within the intermediate result to recover the LF parallax structure. Comprehensive experimental evaluations demonstrate the superiority of our method on both real-world and synthetic LF images when compared with state-of-the-art methods. In addition, we illustrate the benefits and advantages of the proposed approach when applied in various LF-based applications, including image-based rendering and depth estimation enhancement. The code is available at https://github.com/jingjin25/LFASR-FS-GAF.

Light Field Reconstruction with Arbitrary Angular Resolution Using a Deep Coarse-To-Fine Framework

Light Field Reconstruction With A Hybrid Sparse Regularization-Pseudo 4dcnn Framework

Deep Coarse-to-Fine Dense Light Field Reconstruction With Flexible Sampling and Geometry-Aware Fusion

Three Dimensional Reconstruction Using a Lenslet Light Field Camera

Light field angular super-resolution based on structure and scene information

Flexible Light Field Angular Superresolution Via a Deep Coarse-to-Fine Framework

Unsupervised Dense Light Field Reconstruction with Occlusion Awareness

Light Field Reconstruction using Efficient Pseudo 4D Epipolar-Aware Structure

Light field angular super-resolution based on intrinsic and geometric information.

A Coarse-to-Fine Convolutional Neural Network for Light Field Angular Super-Resolution

Micro-Lens Image Stack Upsampling for Densely-Sampled Light Field Reconstruction

Light Field Angular Super-Resolution Via Dense Correspondence Field Reconstruction

Breaking the Spatio-Angular Trade-off for Light Field Super-Resolution via LSTM Modelling on Epipolar Plane Images.

Efficient Light Field Angular Super-Resolution with Sub-Aperture Feature Learning and Macro-Pixel Upsampling.

Reconstructing angular light field by learning spatial features from quadrilateral epipolar geometry

View Position Prior-Supervised Light Field Angular Super-Resolution Network with Asymmetric Feature Extraction and Spatial-Angular Interaction.

Light Field Reconstruction Using Convolutional Network on EPI and Extended Applications

Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network

Epipolar Focus Spectrum: A Novel Light Field Representation and Application in Dense-view Reconstruction

Light Field Reconstruction Using Deep Convolutional Network On Epi

Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses