Abstract:Abstract. The semantic segmentation of the huge amount of acquired 3D data has become an important task in recent years. Meshes have evolved into a standard representation next to Point Clouds (PCs) – not least because of their great visualization possibilities. Compared to PCs, meshes have commonly smaller memory footprints while jointly providing geometrical and high-resolution textural information. For this reason, we opt for semantic mesh segmentation, which is a widely overlooked topic in photogrammetry and remote sensing yet. In this work, we perform an extensive ablation study on multi-modal handcrafted features adapting the Point Cloud Mesh Association (PCMA) (Laupheimer et al., 2020) which establishes explicit connections between faces and points. The multi-modal connections are used in a two-fold manner: (i) to extend per-face descriptors with features engineered on the PC and (ii) to annotate meshes semi-automatically by propagating the manually assigned labels from the PCs. In this way, we derive annotated meshes from the ISPRS benchmark data sets Vaihingen 3D (V3D) and Hessigheim 3D (H3D). To demonstrate the effectiveness of the multi-modal approach, we use well-established and fast Random Forest (RF) models deploying various feature vector compositions and analyze their performances for semantic mesh segmentation. The feature vector compositions consider features derived from the mesh, the PC or both. The results indicate that the combination of radiometric and geometric features outperforms feature sets of a single feature type only. Besides, we observe that relative height is the most crucial feature. The main finding is that the multi-modal feature vector integrates the complementary strengths of the underlying modalities. Whereas the mesh provides outstanding textural information, the dense PCs are superior in geometry. The multi-modal feature descriptor achieves the best performance on both data sets. It significantly outperforms feature sets that incorporate only features derived from the mesh by +7.37 pp and +2.38 pp for mF1 and Overall Accuracy (OA) on V3D. The registered improvement is +9.23 pp and +4.33 pp for mF1 and OA on H3D.

Saliency Regions for 3D Mesh Abstraction

Region-based Saliency Estimation for 3D Shape Analysis and Understanding

Mesh Saliency Detection Via Double Absorbing Markov Chain in Feature Space

Mesh Segmentation with Concavity-Aware Fields

Saliency Guided Subdivision for Single-View Mesh Reconstruction

3D Reconstruction Based on Image Segmentation

3D mesh segmentation using a two-stage merging strategy

A novel segmentation algorithm for complex 3D mesh model in computer vision.

Textured Mesh Saliency: Bridging Geometry and Texture for Human Perception in 3D Graphics

Spectral 3D Mesh Segmentation with a Novel Single Segmentation Field.

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Towards 3D Colored Mesh Saliency: Database and Benchmarks

Perceptual-Saliency Extremum Lines for 3d Shape Illustration

Mesh Segmentation by Local Depth

Spectral 3D mesh segmentation with a novel single segmentation field

Three-dimensional model study on local height

Robust Mesh Segmentation Using Feature-Aware Region Fusion

Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility

MULTI-MODAL SEMANTIC MESH SEGMENTATION IN URBAN SCENES

SAM3D: Segment Anything in 3D Scenes

Towards Mesh Saliency Detection in 6 Degrees of Freedom