View-guided Cost Volume for Light Field Arbitrary-view Disparity Estimation
Rongshan Chen,Hao Sheng,Da Yang,Sizhe Wang,Zhenglong Cui,Ruixuan Cong,Shuai Wang
DOI: https://doi.org/10.1109/TVCG.2024.3453395
2024-09-04
Abstract:Per-view disparity estimation for light field (LF) is critical for various applications such as light field editing, but previous work mostly focuses on estimating disparity for the center view. In this paper, we propose a view-guided cost volume (VGCV), which successfully generates high-quality disparity maps for LF arbitrary view. Unlike previous methods that construct a static cost for center view only, VGCV is designed with view information and can be applicable to arbitrary-view estimation. In particular, since the key to achieving it is to condition cost on view, we extend previous static cost to a conditional one by introducing the spatial and angular information of target view into cost construction and aggregation, experiments show that this way can effectively adapt VGCV to arbitrary-view task. For construction, previous stereo-matching methods usually adopt correlation (e.g., variance) for dynamic estimation, but just using correlation can lose image structure information, which is essential for scene detail recovery, therefore we design an image-guided construction module and use cross-view attention to adapt cost for conditional construction while keeping its spatial information. Then for aggregation, we present a coordinate-guided aggregation module for VGCV regularization, which is specially designed to solve the problem of LF view deviation. Finally, we implement a Light Field Arbitrary-View Disparity Estimation Network (LFAVNet), then perform it on both synthetic and real LFs. Experiments demonstrate that LFAVNet can generate a higher-quality disparity map for arbitrary view in LF. We also extend our method to center-view estimation and light field editing tasks, which all achieve advanced performance.