Improving Three-Dimensional Building Segmentation on Three-Dimensional City Models through Simulated Data and Contextual Analysis for Building Extraction

Frédéric Leroux,Mickaël Germain,Étienne Clabaut,Yacine Bouroubi,Tony St-Pierre

DOI: https://doi.org/10.3390/ijgi13010020

IF: 3.4

2024-01-07

ISPRS International Journal of Geo-Information

Abstract:Digital twins are increasingly gaining popularity as a method for simulating intricate natural and urban environments, with the precise segmentation of 3D objects playing an important role. This study focuses on developing a methodology for extracting buildings from textured 3D meshes, employing the PicassoNet-II semantic segmentation architecture. Additionally, we integrate Markov field-based contextual analysis for post-segmentation assessment and cluster analysis algorithms for building instantiation. Training a model to adapt to diverse datasets necessitates a substantial volume of annotated data, encompassing both real data from Quebec City, Canada, and simulated data from Evermotion and Unreal Engine. The experimental results indicate that incorporating simulated data improves segmentation accuracy, especially for under-represented features, and the DBSCAN algorithm proves effective in extracting isolated buildings. We further show that the model is highly sensible for the method of creating 3D meshes.

geography, physical,remote sensing,computer science, information systems

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address the issue of building segmentation in 3D city models. Specifically, the focus of the research is on:

1. **Method Development**: Proposing a method to extract buildings from textured 3D meshes using the PicassoNet-II semantic segmentation architecture, combined with post-processing evaluation based on Markov random fields (MRF) for contextual analysis.

2. **Data Augmentation**: Increasing the amount of training data by combining simulated data with real data to improve segmentation accuracy. The real data comes from Quebec City, Canada, while the simulated data is sourced from Evermotion and Unreal Engine.

3. **Instance Segmentation**: Applying clustering algorithms (such as DBSCAN) to extract individual building instances from the semantic segmentation results.

4. **Comprehensive Information Utilization**: Considering not only geometric information but also texture information (such as RGB colors) to enhance segmentation performance.

Through these methods, the research aims to improve the accuracy of building segmentation in 3D city models, especially when dealing with objects that have significant color differences but high geometric continuity (e.g., a row of houses with different colored facades). Additionally, the study demonstrates the model's high adaptability in creating 3D meshes.