Abstract:The semantic understanding of indoor 3D point cloud data is crucial for a range of subsequent applications, including indoor service robots, navigation systems, and digital twin engineering. Global features are crucial for achieving high-quality semantic and instance segmentation of indoor point clouds, as they provide essential long-range context information. To this end, we propose JSMNet, which combines a multi-layer network with a global feature self-attention module to jointly segment three-dimensional point cloud semantics and instances. To better express the characteristics of indoor targets, we have designed a multi-resolution feature adaptive fusion module that takes into account the differences in point cloud density caused by varying scanner distances from the target. Additionally, we propose a framework for joint semantic and instance segmentation by integrating semantic and instance features to achieve superior results. We conduct experiments on S3DIS, which is a large three-dimensional indoor point cloud dataset. Our proposed method is compared against other methods, and the results show that it outperforms existing methods in semantic and instance segmentation and provides better results in target local area segmentation. Specifically, our proposed method outperforms PointNet (Qi et al., 2017a) by 16.0% and 26.3% in terms of semantic segmentation mIoU in S3DIS (Area 5) and instance segmentation mPre, respectively. Additionally, it surpasses ASIS (Wang et al., 2019) by 6.0% and 4.6%, respectively, as well as JSPNet (Chen et al., 2022) by a margin of 3.3% for semantic segmentation mIoU and a slight improvement of 0.3% for instance segmentation mPre.

TSPconv-Net: Transformer and Sparse Convolution for 3D Instance Segmentation in Point Clouds

Dynamic Convolution for 3D Point Cloud Instance Segmentation

Superpoint Transformer for 3D Scene Instance Segmentation.

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution

DyCo3D: Robust Instance Segmentation of 3D Point Clouds Through Dynamic Convolution

Spatial Pruned Sparse Convolution for Efficient 3D Object Detection

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

INS-Conv: Incremental Sparse Convolution for Online 3D Segmentation

Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds

SGT-Net: A Transformer-Based Stratified Graph Convolutional Network for 3D Point Cloud Semantic Segmentation

MSTA3D: Multi-scale Twin-attention for 3D Instance Segmentation

CD3IS: Cross Dimensional 3D Instance Segmentation Network for Production Workshop

PointInst3D: Segmenting 3D Instances by Points

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

S3Net: 3D LiDAR Sparse Semantic Segmentation Network

Stratified Transformer for 3D Point Cloud Segmentation

JSMNet Improving Indoor Point Cloud Semantic and Instance Segmentation through Self-Attention and Multiscale

Joint Semantic and Instance Segmentation in 3D Point Cloud Based on Transformer

Video Object Segmentation with 3D Convolution Network

Densely connected graph convolutional network for joint semantic and instance segmentation of indoor point clouds