Uanet: uncertainty-aware cost volume aggregation-based multi-view stereo for 3D reconstruction

Ping Lu,Youcheng Cai,Jiale Yang,Dong Wang,Tingting Wu

DOI: https://doi.org/10.1007/s00371-024-03678-8

IF: 2.835

2024-11-09

The Visual Computer

Abstract:Multi-view stereo (MVS) plays a vital role in 3D reconstruction, which aims to reconstruct the 3D point cloud model from multi-view images. Recently, learning-based MVS methods have demonstrated excellent performance compared with traditional MVS methods. Almost all current learning-based MVS methods focus on improving the accuracy and completeness of the reconstruction results. However, scalability remains a major limitation due to the memory constraint. In this paper, a cascaded network with an uncertainty-aware cost volume aggregation named UANet is proposed for efficient and effective dense 3D reconstruction. In particular, we present a novel uncertainty-aware cost volume aggregation approach that takes pair-wise uncertainty maps as guidance to adaptively aggregate cost volumes. Instead of applying 3D convolutional neural networks (CNNs), we introduce the feature difference with a shallow 2D CNN to compute uncertainty maps, which guarantees both efficiency and effectiveness. Moreover, we adopt a coarse-to-fine strategy and use a group-wise correlation to construct lightweight cost volumes, thus reducing the memory consumption and enabling high-resolution depth map inference. Finally, an uncertainty loss is designed to construct the uncertainty map, which can further boost the performance. Experimental results show that UANet outperforms the previous state-of-the-art methods on three benchmarks of DTU benchmark dataset, Tanks and Temples dataset, and BlendedMVS dataset. Besides, the runtime and memory requirements validate the effectiveness of UANet.

computer science, software engineering