Joint Point Clouds Semantic and Instance Segmentation by Local Aggregation and Clustering

Yun Bai,Qian Xiao,Chaozhi Yang,Yachuan Li,Zongmin Li
DOI: https://doi.org/10.1145/3604078.3604103
2023-01-01
Abstract:Existing methods fuse semantic and instance information directly so that they interfere with each other and cannot accurately identify semantic and instance edges. To address the above issues, we propose a novel network of local aggregation and clustering for joint point clouds semantic and instance segmentation task, named LAC-SIS. First, the point cloud features is extracted by edge convolution, and the geometric descriptors are generated in Euclidean and feature space to aggregate local features. The two geometric descriptors are fused to obtain the local information in two spaces. After that, the semantic and instance feature fusion module is used to selectively fuse the semantic and instance information to obtain the final semantic segmentation result and instance embedding. Finally, the points are clustered in the Euclidean and the feature space to get the instance labels of the global points. The segmentation results on the public datasets S3DIS and ScanNet V2 show that our model effectively improves the ability to distinguish point classes. Our model can accurately predict semantic and instance labels and identifies object edges clearly. Compared with other methods, our method has a significant improvement.
What problem does this paper attempt to address?