Abstract:Existing traditional and ConvNet-based methods for light field depth estimation mainly work on the narrow-baseline scenario. This paper explores the feasibility and capability of ConvNets to estimate depth in another promising scenario: wide-baseline light fields. Due to the deficiency of training samples, a large-scale and diverse synthetic wide-baseline dataset with labelled data is introduced for depth prediction tasks. Considering the practical goal for real-world applications, we design an end-to-end trained lightweight convolutional network to infer depths from light fields, called LLF-Net. The proposed LLF-Net is built by incorporating a cost volume which allows variable angular light field inputs and an attention module that enables to recover details at occlusion areas. Evaluations are made on the synthetic and real-world wide-baseline light fields, and experimental results show that the proposed network achieves the best performance when compared to recent state-of-the-art methods. We also evaluate our LLF-Net on narrow-baseline datasets, and it consequently improves the performance of previous methods.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to perform depth estimation in wide - baseline light fields. Existing traditional methods and methods based on convolutional neural networks (ConvNet) mainly focus on light - field depth estimation in narrow - baseline scenarios. While wide - baseline light fields can provide higher depth accuracy due to their large baselines and high spatial resolutions, there are currently few studies on them. Specifically, the paper addresses the following key issues: 1. **Lack of datasets**: To train a convolutional neural network, a large amount of labeled data is required. However, at that time, there was no large - scale publicly available wide - baseline light - field dataset. Therefore, the author created a large - scale and diverse synthetic wide - baseline light - field dataset (WLF) to support the training and evaluation of the model. 2. **Model design**: The existing convolutional neural network models for wide - baseline light - field depth estimation have a very large number of parameters and are difficult to be applied in practical scenarios, especially in resource - constrained environments such as mobile devices. For this reason, the author designed a lightweight end - to - end trainable convolutional network (LLF - Net), which can reduce the number of parameters while ensuring performance. 3. **Handling occlusion problems**: In wide - baseline light fields, the occlusion problem is a challenge for depth estimation. The author introduced an attention mechanism to handle the detail recovery of occluded areas and maintain depth discontinuities. ### Main contributions of the paper 1. **Dataset construction**: For the first time, a large - scale and diverse synthetic wide - baseline light - field dataset (WLF) was introduced, which contains about 381 light fields, and each light field provides 9×9 angular (RGB) images and a real - depth map. This provides a valuable resource for future research. 2. **Lightweight network design**: A new lightweight end - to - end trainable network (LLF - Net) was proposed. This network has only 1.8 million parameters but performs excellently in the wide - baseline light - field depth - estimation task and outperforms existing methods. 3. **Innovative module**: A new cost - volume module was introduced, which allows flexible light - field inputs and combines an attention mechanism to better handle occlusion problems and maintain depth discontinuities. 4. **Superior performance**: The experimental results show that LLF - Net not only outperforms existing methods in wide - baseline scenarios but also performs excellently in narrow - baseline scenarios. Through these innovations, the paper provides an effective solution for wide - baseline light - field depth estimation and promotes the further development of this field.

A Lightweight Depth Estimation Network for Wide-Baseline Light Fields

A Learning-Based Method Using Epipolar Geometry for Light Field Depth Estimation

Light-field-depth-estimation Network Based on Epipolar Geometry and Image Segmentation.

Zero-Shot Depth Estimation from Light Field Using A Convolutional Neural Network

High precision light field image depth estimation via multi‐region attention enhanced network

Decoupled and Reparameterized Compound Attention-Based Light Field Depth Estimation Network

Depth Estimation Algorithm for Light Field Data by Epipolar Image Analysis and Region Interpolation

LF-DWNet: Robust Depth Estimation Network for Light Field with Disparity Warping

Unsupervised Depth Estimation from Light Field Using a Convolutional Neural Network

LFNAT 2023 Challenge on Light Field Depth Estimation: Methods and Results.

A Light Field Depth Estimation Algorithm by Edge Disparity Extraction

Light Field Depth Estimation Exploiting Linear Structure in EPI

Unsupervised Dense Light Field Reconstruction with Occlusion Awareness

Unsupervised Learning of Light Field Depth Estimation with Spatial and Angular Consistencies

Learning Sub-Pixel Disparity Distribution for Light Field Depth Estimation

LW-Net: A Lightweight Network for Monocular Depth Estimation

Depth Estimation for Light-Field Images Using Stereo Matching and Convolutional Neural Networks

Light Field Depth Estimation via Epipolar Plane Image Analysis and Locally Linear Embedding

Towards Multimodal Depth Estimation from Light Fields

End-to-End Semantic Segmentation Utilizing Multi-scale Baseline Light Field

EPI-based Oriented Relation Networks for Light Field Depth Estimation