DI-MVS: Learning Efficient Multi-View Stereo with Depth-Aware Iterations

Jianfei Jiang,Mingwei Cao,Jun Yi,Chenglong Li

DOI: https://doi.org/10.1109/icassp48485.2024.10446533

2024-01-01

Abstract:Learning-based Multi-View Stereo (MVS) methods aim to reconstruct 3D scenes from a set of 2D calibrated images. However, existing learning-based MVS methods often overlook depth maps that include the geometric shapes of the scene when constructing the cost volume. This can result in suboptimal reconstructions, particularly in low-texture or repetitive-texture regions where valuable geometric information is absent. To address this issue, we develop DI-MVS, a coarse-to-fine framework that effectively incorporates context-guided depth geometry into the cost volume using a depth-aware iterator. First, we employ the proposed depth-aware cost completion module to update the cost volume, followed by 2D ConvGRUs to iteratively optimize depth maps efficiently. Second, we propose a hybrid loss strategy that combines two loss functions’ strengths to improve depth estimation’s robustness. Extensive experiments demonstrate that DI-MVS outperforms state-of-the-art methods on the DTU dataset and the Tanks & Temples benchmark. The source code is available at: https://github.com/JianfeiJ/DI-MVS.