Abstract:Decrease in the cost of acquiring 3D point cloud data coupled with the rapid advancements in GPU computing power have resulted in an increased demand for 3D point cloud semantic segmentation in numerous 3D visual applications, including but not limited to autonomous driving, industrial control, and MR/XR, which further advances the development of deep learning methods in 3D point cloud semantic segmentation. Recently, many novel deep learning network architectures, such as RandLA-Net and Point Transformer, have been proposed and have achieved notable improvements in semantic segmentation accuracy while decreasing the computational load. However, previous research on 3D point cloud semantic segmentation methods has focused primarily on relatively early works, whose approaches have been gradually abandoned over the years and cannot accurately reflect the current research status. Moreover, the existing methods have been categorized based on their input data types, making it difficult to compare the segmentation performance of different techniques and not providing a comprehensive view of the relationship between methods using different network architectures. Therefore, this paper reviews the mainstream 3D semantic segmentation methods developed in the last three years using different deep learning network architectures and is organized into three levels. First, the two principal 3D point cloud data acquisition methods, including their customary datasets and metrics to evaluate model performance, are introduced. Second, a systematic review of 3D semantic segmentation methods based on different network architectures is organized, followed by a statistical analysis of the evaluation of performance between different models on two 3D segmentation datasets—S3DIS and ScanNet. The analysis of model performance on these two commonly used datasets includes model structure relevance, strengths, and limitations. Finally, an insightful discussion of the remaining methodological and application challenges and potential research directions is provided. This paper offers an extensive overview of the recent three-year research progress in 3D point cloud semantic segmentation and summarizes various network architecture pipelines, elucidates their fundamental operations, compares the model performance across multiple architectures, discusses their notable strengths and limitations, most importantly, concludes the current challenges and promising research directions for future investigations. Furthermore, this paper enables researchers to effortlessly identify the relevant research and research hotspots among different 3D point cloud semantic segmentation methods based on the analyses presented and aims to update the reviews on 3D point cloud semantic segmentation methods with a better viewpoint and highlight key properties and contributions of proposed methods, providing promising research directions for the main challenges.

A survey of large-scale point cloud semantic segmentation based on deep learning

Deep-Learning-Based Point Cloud Semantic Segmentation: A Survey

3D Semantic Segmentation Using Deep Learning for Large-Scale Indoor Point Cloud

Deep Learning Based Point Cloud Segmentation: A Survey

Deep learning-based semantic segmentation of three-dimensional point cloud: a comprehensive review

3D Point Cloud Semantic Segmentation: State of the Art and Challenges

Research Progress Analysis of Point Cloud Segmentation Based on Deep Learning

A Point-Based Deep Learning Network for Semantic Segmentation of MLS Point Clouds

3D Semantic Segmentation of Large-Scale Point-Clouds in Urban Areas Using Deep Learning

A Deep Learning Network for Semantic Labeling of Large-Scale Urban Point Clouds

Exploring Semantic Information Extraction From Different Data Forms in 3D Point Cloud Semantic Segmentation

A review of deep learning methods for semantic segmentation of remote sensing imagery

A Brief Survey on Semantic Segmentation with Deep Learning

LASDU: A Large-Scale Aerial LiDAR Dataset for Semantic Labeling in Dense Urban Areas.

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

CSPC-Dataset: New LiDAR Point Cloud Dataset and Benchmark for Large-Scale Scene Semantic Segmentation

A Semantic Segmentation Approach Based On Deeplab Network In High-Resolution Remote Sensing Images

A review of point cloud segmentation for understanding 3D indoor scenes

A comprehensive overview of deep learning techniques for 3D point cloud classification and semantic segmentation

Deep Learning on Point Clouds and Its Application: A Survey