Abstract:In this work, we address the challenging task of few-shot and zero-shot 3D point cloud semantic segmentation. The success of few-shot semantic segmentation in 2D computer vision is mainly driven by the pre-training on large-scale datasets like imagenet. The feature extractor pre-trained on large-scale 2D datasets greatly helps the 2D few-shot learning. However, the development of 3D deep learning is hindered by the limited volume and instance modality of datasets due to the significant cost of 3D data collection and annotation. This results in less representative features and large intra-class feature variation for few-shot 3D point cloud segmentation. As a consequence, directly extending existing popular prototypical methods of 2D few-shot classification/segmentation into 3D point cloud segmentation won't work as well as in 2D domain. To address this issue, we propose a Query-Guided Prototype Adaption (QGPA) module to adapt the prototype from support point clouds feature space to query point clouds feature space. With such prototype adaption, we greatly alleviate the issue of large feature intra-class variation in point cloud and significantly improve the performance of few-shot 3D segmentation. Besides, to enhance the representation of prototypes, we introduce a Self-Reconstruction (SR) module that enables prototype to reconstruct the support mask as well as possible. Moreover, we further consider zero-shot 3D point cloud semantic segmentation where there is no support sample. To this end, we introduce category words as semantic information and propose a semantic-visual projection model to bridge the semantic and visual spaces. Our proposed method surpasses state-of-the-art algorithms by a considerable 7.90% and 14.82% under the 2-way 1-shot setting on S3DIS and ScanNet benchmarks, respectively.

Noisy Few-shot 3D Point Cloud Scene Segmentation.

Robust Few-Shot 3D Point Cloud Scene Segmentation

Towards Robust Few-shot Point Cloud Semantic Segmentation

Pass3d: Precise And Accelerated Semantic Segmentation For 3d Point Cloud

Few-shot 3D Point Cloud Semantic Segmentation

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

A Simple Framework of Few-Shot Learning Using Sparse Annotations for Semantic Segmentation of 3-D Point Clouds

Blessing Few-Shot Segmentation Via Semi-Supervised Learning with Noisy Support Images

Generalized Few-Shot Point Cloud Segmentation Via Geometric Words

Information Gap Narrowing for Point Cloud Few-shot Segmentation

Iterative Few-shot Semantic Segmentation from Image Label Text

Rethinking Few-shot 3D Point Cloud Semantic Segmentation

Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation

No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentation

Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement

CAM/CAD Point Cloud Part Segmentation via Few-Shot Learning

Few-shot 3D LiDAR Semantic Segmentation for Autonomous Driving

Less is More: Towards Efficient Few-shot 3D Semantic Segmentation via Training-free Networks

SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds with 1000x Fewer Labels

Few-shot Point Cloud Semantic Segmentation Via Support-Query Feature Interaction

Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation