Abstract:The semantic understanding of indoor 3D point cloud data is crucial for a range of subsequent applications, including indoor service robots, navigation systems, and digital twin engineering. Global features are crucial for achieving high-quality semantic and instance segmentation of indoor point clouds, as they provide essential long-range context information. To this end, we propose JSMNet, which combines a multi-layer network with a global feature self-attention module to jointly segment three-dimensional point cloud semantics and instances. To better express the characteristics of indoor targets, we have designed a multi-resolution feature adaptive fusion module that takes into account the differences in point cloud density caused by varying scanner distances from the target. Additionally, we propose a framework for joint semantic and instance segmentation by integrating semantic and instance features to achieve superior results. We conduct experiments on S3DIS, which is a large three-dimensional indoor point cloud dataset. Our proposed method is compared against other methods, and the results show that it outperforms existing methods in semantic and instance segmentation and provides better results in target local area segmentation. Specifically, our proposed method outperforms PointNet (Qi et al., 2017a) by 16.0% and 26.3% in terms of semantic segmentation mIoU in S3DIS (Area 5) and instance segmentation mPre, respectively. Additionally, it surpasses ASIS (Wang et al., 2019) by 6.0% and 4.6%, respectively, as well as JSPNet (Chen et al., 2022) by a margin of 3.3% for semantic segmentation mIoU and a slight improvement of 0.3% for instance segmentation mPre.

JSPNet: Learning Joint Semantic & Instance Segmentation of Point Clouds Via Feature Self-Similarity and Cross-Task Probability

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

Joint Semantic-Instance Segmentation of 3D Point Clouds: Instance Separation and Semantic Fusion.

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds

Self-Prediction for Joint Instance and Semantic Segmentation of Point Clouds

JSMNet Improving Indoor Point Cloud Semantic and Instance Segmentation through Self-Attention and Multiscale

Joint Semantic Segmentation using representations of LiDAR point clouds and camera images

Associatively Segmenting Instances and Semantics in Point Clouds

Joint Semantic-Instance Segmentation Method for Intelligent Transportation System

3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection

Joint Object Segmentation and Depth Upsampling

PointMM: Point Cloud Semantic Segmentation CNN under Multi-Spatial Feature Encoding and Multi-Head Attention Pooling

MSIDA-Net: Point Cloud Semantic Segmentation via Multi-Spatial Information and Dual Adaptive Blocks

Robust 3D Semantic Segmentation Method Based on Multi-Modal Collaborative Learning

SEMANTIC SEGMENTATION OF INDOOR 3D POINT CLOUDS BY JOINT OPTIMIZATION OF GEOMETRIC FEATURES AND NEURAL NETWORKS

SIESEF-FusionNet: Spatial Inter-correlation Enhancement and Spatially-Embedded Feature Fusion Network for LiDAR Point Cloud Semantic Segmentation

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

Semantic Segmentation of Urban Airborne LiDAR Point Clouds Based on Fusion Attention Mechanism and Multi-Scale Features

BEMF-Net: Semantic Segmentation of Large-Scale Point Clouds via Bilateral Neighbor Enhancement and Multi-Scale Fusion