Full-automatic high-precision scene 3D reconstruction method with water-area intelligent complementation and mesh optimization for UAV images

Bingxuan Guo,Yingwei Ge,Xiongwu Xiao,Chao Wang,Jianya Gong,Deren Li

DOI: https://doi.org/10.1080/17538947.2024.2317441

IF: 4.606

2024-02-18

International Journal of Digital Earth

Abstract:Fast and high-precision urban scene 3D modeling is the foundational data infrastructure for the digital earth and smart cities. However, due to challenges such as water-area matching difficulties and issues like data redundancy and insufficient observations, existing full-automatic 3D modeling methods often result in water-area missing and many small holes in the models and insufficient local-model accuracy. To overcome these challenges, full-automatic high-precision scene 3D reconstruction method with water-area intelligent complementation on depth maps and mesh optimization is proposed. Firstly, SfM was used to calculated image poses and PatchMatch was used to generated initial depth maps. Secondly, a simplified GAN extracted water-area masks and ray tracing was used achieve high-precision auto-completed water-area depth values. Thirdly, fully connected CRF optimized water-areas and arounds in depth maps. Fourthly, high-precision 3D point clouds were obtained using depth map fusion based on clustering culling and depth least squares. Then, mesh was generated and optimized using similarity measurement and vertex gradients to obtain refined mesh. Finally, high-precision scene 3D models without water-area missing or holes were generated. The results showed that: to compare with the-state-of-art ContextCapture, the proposed method enhances model completeness by 14.3%, raises average accuracy by 14.5% and improves processing efficiency by 63.6%.

geography, physical,remote sensing

What problem does this paper attempt to address?

The problem this paper attempts to address is the rapid and high-precision 3D modeling of urban scenes. Specifically, existing fully automatic 3D modeling methods encounter the following issues when dealing with water areas:

1. **Missing Water Areas**: Due to the weak texture of water, existing 3D modeling methods find it difficult to extract feature points, resulting in water areas often being missing or having holes in the model.
2. **Insufficient Local Model Accuracy**: Weak texture areas like water are difficult to match, leading to low local model accuracy.
3. **Low Modeling Efficiency**: When processing large-scale or complex scenes, some modeling software or algorithms run slowly, affecting overall modeling efficiency.

To solve these problems, this study proposes a fully automatic high-precision scene 3D reconstruction method that combines intelligent water completion and mesh optimization techniques. The specific steps are as follows:

1. Use a simplified Generative Adversarial Network (GAN) to extract water masks and combine ray tracing technology to achieve high-precision automatic completion of water depth values.
2. Optimize the entire depth map using Conditional Random Fields (CRF).
3. Improve model accuracy through depth map fusion and mesh refinement methods based on similarity measurement and vertex gradient optimization.
4. Finally, generate a high-precision 3D model without missing water areas or holes.

Compared with existing methods, this method improves model integrity, average accuracy, and processing efficiency by 14.3%, 14.5%, and 63.6%, respectively.