Abstract:In this work, we address the challenging task of few-shot and zero-shot 3D point cloud semantic segmentation. The success of few-shot semantic segmentation in 2D computer vision is mainly driven by the pre-training on large-scale datasets like imagenet. The feature extractor pre-trained on large-scale 2D datasets greatly helps the 2D few-shot learning. However, the development of 3D deep learning is hindered by the limited volume and instance modality of datasets due to the significant cost of 3D data collection and annotation. This results in less representative features and large intra-class feature variation for few-shot 3D point cloud segmentation. As a consequence, directly extending existing popular prototypical methods of 2D few-shot classification/segmentation into 3D point cloud segmentation won't work as well as in 2D domain. To address this issue, we propose a Query-Guided Prototype Adaption (QGPA) module to adapt the prototype from support point clouds feature space to query point clouds feature space. With such prototype adaption, we greatly alleviate the issue of large feature intra-class variation in point cloud and significantly improve the performance of few-shot 3D segmentation. Besides, to enhance the representation of prototypes, we introduce a Self-Reconstruction (SR) module that enables prototype to reconstruct the support mask as well as possible. Moreover, we further consider zero-shot 3D point cloud semantic segmentation where there is no support sample. To this end, we introduce category words as semantic information and propose a semantic-visual projection model to bridge the semantic and visual spaces. Our proposed method surpasses state-of-the-art algorithms by a considerable 7.90% and 14.82% under the 2-way 1-shot setting on S3DIS and ScanNet benchmarks, respectively.

Few-shot 3D Point Cloud Semantic Segmentation

Robust Few-Shot 3D Point Cloud Scene Segmentation

Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation

Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Network

Compositional Prototype Network with Multi-view Comparision for Few-Shot Point Cloud Semantic Segmentation

Prototype Queue Learning for Multi-Class Few-Shot Semantic Segmentation

Noisy Few-shot 3D Point Cloud Scene Segmentation.

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds

Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement

Few-shot Point Cloud Semantic Segmentation Via Support-Query Feature Interaction

Dynamic Prototype Adaptation with Distillation for Few-shot Point Cloud Segmentation

Information Gap Narrowing for Point Cloud Few-shot Segmentation

Generalized Few-Shot Point Cloud Segmentation Via Geometric Words

Towards Robust Few-shot Point Cloud Semantic Segmentation

APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic Segmentation

Rethinking Few-shot 3D Point Cloud Semantic Segmentation

Cross-Class Bias Rectification for Point Cloud Few-Shot Segmentation

Iterative Few-shot Semantic Segmentation from Image Label Text

A Simple Framework of Few-Shot Learning Using Sparse Annotations for Semantic Segmentation of 3-D Point Clouds

TPSN: Transformer-based Multi-Prototype Search Network for Few-Shot Semantic Segmentation

Multiscale Attention-Based Prototypical Network for Few-Shot Semantic Segmentation.