Abstract:Despite the significant recent progress made on 3D point cloud semantic segmentation, the current methods require training data for all classes at once, and are not suitable for real-life scenarios where new categories are being continuously discovered. Substantial memory storage and expensive re-training is required to update the model to sequentially arriving data for new concepts. In this paper, to continually learn new categories using previous knowledge, we introduce class-incremental semantic segmentation of 3D point cloud. Unlike 2D images, 3D point clouds are disordered and unstructured, making it difficult to store and transfer knowledge especially when the previous data is not available. We further face the challenge of semantic shift, where previous/future classes are indiscriminately collapsed and treated as the background in the current step, causing a dramatic performance drop on past classes. We exploit the structure of point cloud and propose two strategies to address these challenges. First, we design a geometry-aware distillation module that transfers point-wise feature associations in terms of their geometric characteristics. To counter forgetting caused by the semantic shift, we further develop an uncertainty-aware pseudo-labelling scheme that eliminates noise in uncertain pseudo-labels by label propagation within a local neighborhood. Our extensive experiments on S3DIS and ScanNet in a class-incremental setting show impressive results comparable to the joint training strategy (upper bound). Code is available at: https://github.com/leolyj/3DPC-CISS

Associatively Segmenting Instances and Semantics in Point Clouds

Pass3d: Precise And Accelerated Semantic Segmentation For 3d Point Cloud

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

Semantic Graph Based Place Recognition for 3D Point Clouds.

Joint Semantic-Instance Segmentation of 3D Point Clouds: Instance Separation and Semantic Fusion.

Semantic Labeling and Instance Segmentation of 3D Point Clouds Using Patch Context Analysis and Multiscale Processing

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

Joint Point Clouds Semantic and Instance Segmentation by Local Aggregation and Clustering

Affinity3D: Propagating Instance-Level Semantic Affinity for Zero-Shot Point Cloud Semantic Segmentation

Exploring Semantic Information Extraction From Different Data Forms in 3D Point Cloud Semantic Segmentation

Instance-Aware Embedding for Point Cloud Instance Segmentation

Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation

Hierarchical Aggregation for 3D Instance Segmentation

FreePoint: Unsupervised Point Cloud Instance Segmentation

Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation

3DCFS: Fast and Robust Joint 3D Semantic-Instance Segmentation via Coupled Feature Selection

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation

PointGS: Bridging and Fusing Geometric and Semantic Space for 3D Point Cloud Analysis