Embedded Shape Matching in Photogrammetry Data for Modeling Making Knowledge

Demircan Tas,Mine Özkar

2023-12-21

Abstract:In three-dimensional models obtained by photogrammetry of existing structures, all of the shapes that the eye can select cannot always find their equivalents in the geometric components of the model. However, the matching of meaningful parts and assemblages with the records acquired with rapid and detailed documentation methods will provide an advantage for the creation of information models of existing structures. While aiming to produce answers to this problem and in order to overcome the difficulties of pattern recognition in three-dimensional models, we used two-dimensional samples obtained by projection. Processing techniques such as ambient occlusion, curvature and normal maps are commonly used in modern computer graphics applications that enable the representation of three-dimensional surface properties in two-dimensional data sets. The method we propose is based on the recognition of patterns through these mappings instead of the usual light-based visualization. The first stage of the application is photogrammetric capture of a few examples of Zeugma mosaics and three-dimensional digital modeling of a set of Seljuk era brick walls based on knowledge obtained through architectural history literature. The second stage covers the creation of digital models byprocessing the surface representation obtained from this data using Alice Vision, OpenCV-Python, and Autodesk Maya to include information on aspects of the making of the walls. What is envisioned for the next stages is that the mapping data contributes and supports the knowledge for rule-based design and making processesof cultural heritage.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem that not all visible shapes can be fully reflected in geometric components when using **3D models** (obtained through **photogrammetry techniques**) to represent existing buildings. Specifically, the author points out:

1. **Lack of geometric information**: Although existing 3D models have a high level of detail, they lack part - whole relationships and attribute data, which makes them difficult to be directly used in the design and analysis process.

2. **Challenges in pattern recognition**: There are difficulties in pattern recognition in 3D models. Therefore, the author proposes a method based on 2D projection samples to simplify the pattern recognition process.

3. **Creation of information models**: In order to better record and understand existing buildings, it is necessary to match meaningful parts with the records obtained through rapid and detailed documentation methods, thus providing advantages for creating information models.

4. **Embedding of manufacturing knowledge**: The author hopes to develop a form - matching method that can extract and model manufacturing knowledge from photogrammetric data, especially for data sets of damaged structures or data lost during the scanning process, and can complete the supplementation of missing data or parts under the guidance of high - level patterns.

### Overview of the method

To achieve the above goals, the author proposes the following steps:

- **Data collection**: Use photogrammetry techniques to obtain 3D models of historical mosaics and brick walls.
- **Image processing**: Use tools such as Alice Vision, OpenCV - Python and Autodesk Maya to process these models and generate 2D maps such as ambient occlusion, curvature and normal maps.
- **Pattern recognition**: Train through HAAR - cascade classifiers to identify specific patterns (such as bricks) in 2D maps and locate the positions of these patterns in 3D models.

Finally, this method aims to support the application of computational design and manufacturing rules, especially in the process of restoring and reconstructing historical buildings, and can more effectively extract and utilize manufacturing knowledge.