Abstract:Although extensive research has been conducted on 3D point cloud segmentation, effectively adapting generic models to novel categories remains a formidable challenge. This paper proposes a novel approach to improve point cloud few-shot segmentation (PC-FSS) models. Unlike existing PC-FSS methods that directly utilize categorical information from support prototypes to recognize novel classes in query samples, our method identifies two critical aspects that substantially enhance model performance by reducing contextual gaps between support prototypes and query features. Specifically, we (1) adapt support background prototypes to match query context while removing extraneous cues that may obscure foreground and background in query samples, and (2) holistically rectify support prototypes under the guidance of query features to emulate the latter having no semantic gap to the query targets. Our proposed designs are agnostic to the feature extractor, rendering them readily applicable to any prototype-based methods. The experimental results on S3DIS and ScanNet demonstrate notable practical benefits, as our approach achieves significant improvements while still maintaining high efficiency. The code for our approach is available at <a class="link-external link-https" href="https://github.com/AaronNZH/Boosting-Few-shot-3D-Point-Cloud-Segmentation-via-Query-Guided-Enhancement" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

The paper primarily proposes a new method for 3D point cloud segmentation in few-shot (small sample) scenarios, aiming to improve the model's performance when handling new categories. Specifically, the paper addresses the following core issues: 1. **Consistency Issue of Background and Foreground Representation**: Existing methods directly use the category information from support prototypes to identify new categories in query samples, but they overlook the content differences between the query point cloud and the support point cloud. This may lead to the background representation being biased towards the context clues of the support samples, thereby affecting the model's accurate identification of the target category in the query point cloud. 2. **Semantic Gap Issue**: Due to scene variations between the query point cloud and the support point cloud, even in the foreground region, the support prototypes (especially the background prototypes) may not match the query features well, leading to a decline in model performance. To address the above issues, the paper proposes two key steps: - **Background Prototype Adaptation (BPA)**: By adjusting the support background prototypes to match the query context, it removes additional clues that may confuse the foreground and background, thereby improving the background differentiation capability. - **Holistic Rectification (HR)**: Optimizing the support prototypes under guidance to better adapt to the query context, thereby narrowing the semantic gap between the query features and the support prototypes. Through the above two steps, the proposed solution effectively reduces the semantic gap between the query features and the support prototypes, improving the model's performance in few-shot scenarios. Additionally, these improvements do not rely on specific feature extractors and can be easily applied to any prototype-based point cloud few-shot segmentation (PC-FSS) framework. The experiments show significant performance improvements while maintaining high efficiency.

Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement

Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation

Few-shot Point Cloud Semantic Segmentation Via Support-Query Feature Interaction

Rethinking Few-shot 3D Point Cloud Semantic Segmentation

Generalized Few-Shot Point Cloud Segmentation Via Geometric Words

Few-shot 3D Point Cloud Semantic Segmentation

Dynamic Prototype Adaptation with Distillation for Few-shot Point Cloud Segmentation

Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation

Cross-Class Bias Rectification for Point Cloud Few-Shot Segmentation

Query-guided Support Prototypes for Few-shot 3D Indoor Segmentation

Boosting Few-Shot Segmentation via Instance-Aware Data Augmentation and Local Consensus Guided Cross Attention

Robust Few-Shot 3D Point Cloud Scene Segmentation

Localization and Expansion: A Decoupled Framework for Point Cloud Few-shot Semantic Segmentation

Multi-Similarity Enhancement Network for Few-Shot Segmentation.

Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Network

Towards Robust Few-shot Point Cloud Semantic Segmentation

Aggregation and Purification: Dual Enhancement Network for Point Cloud Few-shot Segmentation

Tackling background ambiguities in multi-class few-shot point cloud semantic segmentation

Multimodality Helps Few-Shot 3D Point Cloud Semantic Segmentation

Information Gap Narrowing for Point Cloud Few-shot Segmentation

Superpoint-guided Semi-supervised Semantic Segmentation of 3D Point Clouds