Abstract:Abstract. The semantic segmentation of the huge amount of acquired 3D data has become an important task in recent years. Meshes have evolved into a standard representation next to Point Clouds (PCs) – not least because of their great visualization possibilities. Compared to PCs, meshes have commonly smaller memory footprints while jointly providing geometrical and high-resolution textural information. For this reason, we opt for semantic mesh segmentation, which is a widely overlooked topic in photogrammetry and remote sensing yet. In this work, we perform an extensive ablation study on multi-modal handcrafted features adapting the Point Cloud Mesh Association (PCMA) (Laupheimer et al., 2020) which establishes explicit connections between faces and points. The multi-modal connections are used in a two-fold manner: (i) to extend per-face descriptors with features engineered on the PC and (ii) to annotate meshes semi-automatically by propagating the manually assigned labels from the PCs. In this way, we derive annotated meshes from the ISPRS benchmark data sets Vaihingen 3D (V3D) and Hessigheim 3D (H3D). To demonstrate the effectiveness of the multi-modal approach, we use well-established and fast Random Forest (RF) models deploying various feature vector compositions and analyze their performances for semantic mesh segmentation. The feature vector compositions consider features derived from the mesh, the PC or both. The results indicate that the combination of radiometric and geometric features outperforms feature sets of a single feature type only. Besides, we observe that relative height is the most crucial feature. The main finding is that the multi-modal feature vector integrates the complementary strengths of the underlying modalities. Whereas the mesh provides outstanding textural information, the dense PCs are superior in geometry. The multi-modal feature descriptor achieves the best performance on both data sets. It significantly outperforms feature sets that incorporate only features derived from the mesh by +7.37 pp and +2.38 pp for mF1 and Overall Accuracy (OA) on V3D. The registered improvement is +9.23 pp and +4.33 pp for mF1 and OA on H3D.

SUM: A Benchmark Dataset of Semantic Urban Meshes

Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges

LASDU: A Large-Scale Aerial LiDAR Dataset for Semantic Labeling in Dense Urban Areas.

CUS3D: A New Comprehensive Urban-Scale Semantic-Segmentation 3D Benchmark Dataset

SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds

Deep Learning-Based Semantic Segmentation of Urban-Scale 3D Meshes in Remote Sensing: A Survey

A Texture Integrated Deep Neural Network for Semantic Segmentation of Urban Meshes

A Surface Graph Based Deep Learning Framework for Large-Scale Urban Mesh Semantic Segmentation

WHU-Urban3D: An urban scene LiDAR point cloud dataset for semantic instance segmentation

Annotation Tool and Urban Dataset for 3D Point Cloud Semantic Segmentation

MULTI-MODAL SEMANTIC MESH SEGMENTATION IN URBAN SCENES

Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds

CSPC-Dataset: New LiDAR Point Cloud Dataset and Benchmark for Large-Scale Scene Semantic Segmentation

HRHD-HK: A benchmark dataset of high-rise and high-density urban scenes for 3D semantic segmentation of photogrammetric point clouds

Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks

GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data

UrbanLF: A Comprehensive Light Field Dataset for Semantic Segmentation of Urban Scenes

Mesh-Based DGCNN: Semantic Segmentation of Textured 3-D Urban Scenes

An Aerial Photogrammetry Benchmark Dataset for Point Cloud Segmentation and Style Translation

A 3D INDOOR-OUTDOOR BENCHMARK DATASET FOR LoD3 BUILDING POINT CLOUD SEMANTIC SEGMENTATION

UrbanSegNet: An urban meshes semantic segmentation network using diffusion perceptron and vertex spatial attention