Abstract:This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple lowresolution cameras. The performance of existing methods is still limited, as they produce either blurry results on plain textured areas or distortions around depth discontinuous boundaries. To tackle this challenge, we propose a novel end-to-end learningbased approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation, while the other module warps another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations adaptively via the learned attention maps, leading to the final high-resolution LF image with satisfactory results on both plain textured areas and depth discontinuous boundaries. Besides, to promote the effectiveness of our method trained with simulated hybrid data on real hybrid data captured by a hybrid LF imaging system, we carefully design the network architecture and the training strategy. Extensive experiments on both real and simulated hybrid data demonstrate the significant superiority of our approach over state-of-the-art ones. To the best of our knowledge, this is the first end-to-end deep learning method for LF reconstruction from a real hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and benefit LF data storage and transmission. The code will be publicly available at https://github.com/jingjin25/LFhybridSR-Fusion.

The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring

Three Dimensional Reconstruction Using a Lenslet Light Field Camera

High-Resolution Light Field Cameras Based on A Hybrid Imaging System

Light field angular super-resolution based on intrinsic and geometric information.

Simulation Of Imaging And Refocusing For Light Field Photography

Design of a Novel Microlens Array and Imaging System for Light Fields

The Light Field 3D Scanner.

Light Field Stitching for Extended Synthetic Aperture

Variable Aperture Light Field Photography: Overcoming the Diffraction-Limited Spatio-Angular Resolution Tradeoff.

A Wide Field-of-View Light-Field Camera with Adjustable Multiplicity for Practical Applications

Light field camera all-in-focus image acquisition based on angular information

Light Field Measurement Based On The Single-Lens Coherent Diffraction Imaging

10‐2: A Deep Depth of Field Near Eye Light Field Displays Utilizing LC Lens and Dual‐layer LCDs

Light Field All-in-focus Image Fusion Based on Spatially-Guided Angular Information.

Camera Array Based Light Field Microscopy

High-Resolution Light Field Capture with Coded Aperture

Light Field Spatial Resolution Enhancement Framework

Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses

Light-Field Raw Data Synthesis from RGB-D Images: Pushing to the Extreme

Light Field Sensor and Real-Time Panorama Imaging Multi-Camera System and the Design of Data Acquisition

Light Field Reconstruction Via Attention-Guided Deep Fusion of Hybrid Lenses