Abstract:Multi-view stereo plays an important role in 3D reconstruction but suffers from low reconstruction efficiency and has difficulties reconstructing areas with low or repeated textures. To address this, we propose MVP-Stereo, a novel multi-view parallel patchmatch stereo method. MVP-Stereo employs two key techniques. First, MVP-Stereo utilizes multi-view dilated ZNCC to handle low texture and repeated texture by dynamically adjusting the matching window size based on image variance and using a portion of pixels to calculate matching costs without increasing computational complexity. Second, MVP-Stereo leverages multi-scale parallel patchmatch to reconstruct the depth map for each image in a highly efficient manner, which is implemented by CUDA with random initialization, multi-scale parallel spatial propagation, random refinement, and the coarse-to-fine strategy. Experiments on the Strecha dataset, the ETH3D benchmark, and the UAV dataset demonstrate that MVP-Stereo can achieve competitive reconstruction quality compared to state-of-the-art methods with the highest reconstruction efficiency. For example, MVP-Stereo outperforms COLMAP in reconstruction quality by around 30% of reconstruction time, and achieves around 90% of the quality of ACMMP and SD-MVS in only around 20% of the time. In summary, MVP-Stereo can efficiently reconstruct high-quality point clouds and meet the requirements of several photogrammetric applications, such as emergency relief, infrastructure inspection, and environmental monitoring.
environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
What problem does this paper attempt to address?
This paper aims to solve two main problems of Multi - View Stereo (MVS) in 3D reconstruction: low reconstruction efficiency and difficulty in reconstructing low - texture or repetitive - texture areas. Specifically, traditional MVS methods have high computational complexity when dealing with large - scale scenes and have poor reconstruction effects on low - texture or repetitive - texture areas. To solve these problems, the author proposes MVP - Stereo, a new multi - view parallel PatchMatch stereo method.
### Main Contributions
1. **Multi - View Dilated Zero - Mean Normalized Cross - Correlation (Multi - View Dilated ZNCC)**
- Introduced Multi - View Dilated Zero - Mean Normalized Cross - Correlation (Multi - View Dilated ZNCC), which adjusts the matching window size dynamically to handle low - texture and repetitive - texture areas while avoiding increasing computational complexity. This method utilizes the idea of dilated convolution and calculates the matching cost through some pixels, thereby expanding the receptive field without increasing computational complexity.
2. **Multi - Scale Parallel PatchMatch**
- Proposed a multi - scale parallel PatchMatch method, which is implemented on GPU and includes random initialization, multi - scale parallel spatial propagation, random refinement, and a coarse - to - fine strategy. Different from the method of directly performing parallel spatial propagation, MVP - Stereo proposes multi - scale parallel spatial propagation, divides pixels into blocks of different scales, and propagates the best plane parameters in each block to accelerate the convergence speed.
3. **Efficient 3D Reconstruction**
- MVP - Stereo can significantly improve the reconstruction efficiency while maintaining high - quality reconstruction. Experimental results show that MVP - Stereo outperforms existing advanced methods, such as COLMAP, ACMMP, and SD - MVS, on multiple datasets, especially in terms of reconstruction quality and time efficiency.
### Experimental Verification
To verify the effectiveness of MVP - Stereo, the author conducted qualitative and quantitative experiments on the Strecha dataset, the ETH3D benchmark dataset, and a UAV dataset. The experimental results show that MVP - Stereo can not only efficiently generate high - quality point clouds but also meet the needs of various photogrammetric applications, such as emergency rescue, infrastructure inspection, and environmental monitoring.
### Formula Presentation
- **Multi - View Dilated ZNCC**
\[
\text{ZNCC}(I_S, I_N, p, f_p)=\frac{\sum_{j\in W_p}(I_S(j)-\bar{I_S})(I_N(f_p(j, I_N))-\bar{I_N})}{\sqrt{\sum_{j\in W_p}(I_S(j)-\bar{I_S})^2\sum_{j\in W_p}(I_N(f_p(j, I_N))-\bar{I_N})^2}}
\]
- **Dynamic Window Adjustment Strategy**
\[
\text{ZNCC}(I_S, p, f_p)=(\beta r_p+(1 - \beta)r_p)\text{ZNCC}(I_S, p, f_p)
\]
where \(\sigma_p\) is the variance of pixel values in the window, \(\beta=\exp(-\max(\sigma_p/T_\sigma - 1,0))\), and when \(\sigma_p < T_\sigma = 0.005\), the area is considered to be a low - texture or repetitive - texture area.
### Summary
MVP - Stereo effectively solves the shortcomings of traditional MVS methods in dealing with low - texture and repetitive - texture areas by introducing Multi - View Dilated ZNCC and Multi - Scale Parallel PatchMatch methods, and at the same time significantly improves the reconstruction efficiency. These innovations make MVP - Stereo have broad application prospects in various application scenarios.