LNMVSNet: A Low-Noise Multi-View Stereo Depth Inference Method for 3D Reconstruction

Weiming Luo,Zongqing Lu,Qingmin Liao

DOI: https://doi.org/10.3390/s24082400

IF: 3.9

2024-04-10

Sensors

Abstract:With the widespread adoption of modern RGB cameras, an abundance of RGB images is available everywhere. Therefore, multi-view stereo (MVS) 3D reconstruction has been extensively applied across various fields because of its cost-effectiveness and accessibility, which involves multi-view depth estimation and stereo matching algorithms. However, MVS tasks face noise challenges because of natural multiplicative noise and negative gain in algorithms, which reduce the quality and accuracy of the generated models and depth maps. Traditional MVS methods often struggle with noise, relying on assumptions that do not always hold true under real-world conditions, while deep learning-based MVS approaches tend to suffer from high noise sensitivity. To overcome these challenges, we introduce LNMVSNet, a deep learning network designed to enhance local feature attention and fuse features across different scales, aiming for low-noise, high-precision MVS 3D reconstruction. Through extensive evaluation of multiple benchmark datasets, LNMVSNet has demonstrated its superior performance, showcasing its ability to improve reconstruction accuracy and completeness, especially in the recovery of fine details and clear feature delineation. This advancement brings hope for the widespread application of MVS, ranging from precise industrial part inspection to the creation of immersive virtual environments.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

This paper attempts to address the noise problem in multi-view stereo (MVS) 3D reconstruction. Specifically, MVS tasks face challenges such as natural multiplicative noise and algorithmic negative gain in practical applications, which reduce the quality and accuracy of generated models and depth maps. Traditional MVS methods often rely on assumptions that do not always hold under real-world conditions, while deep learning-based MVS methods are highly sensitive to noise. To overcome these issues, the authors propose LNMVSNet, a deep learning network designed to enhance local feature attention and fuse features at different scales, aiming to achieve low-noise, high-precision MVS 3D reconstruction.

The main contributions of the paper include:
1. Proposing LNMVSNet, which effectively utilizes the different sensitivities to noise and pixel weight scoring of multi-level features by introducing a multi-level deep feature fusion mechanism and a novel attention filtering mechanism, thereby reducing the impact of noise during the initial depth estimation stage.
2. Conducting extensive evaluations on multiple benchmark datasets, LNMVSNet demonstrates its superior performance, generating smooth and low-noise depth estimations and point clouds. Additionally, the impact of noise on reconstruction evaluation metrics is analyzed through qualitative experimental results.