Abstract:Zero-shot semantic segmentation aims to segment novel classes that have not been encountered during the training phase. Existing methods leverage available text features obtained from pretrained language models to produce semantic segmentation results for both base and novel classes. However, the text-based feature-producing paradigm only provides insufficient class correlations and limits the full exploitation of image features from base classes. Besides, there exists a non-negligible domain gap between the text and image domains, resulting in severe feature bias during feature production. Different from existing methods, we advance the zero-shot semantic segmentation through attribute correlations. Specifically, we introduce a set of shared-attribute labels, of which the design fully considers the structural relations between attributes and classes, to provide rational and sufficient attribute-class correlations. Besides, due to the minor intra-class variations of shared attributes, the text features are more easily mapped to image features, thereby alleviating the domain gap issue. Furthermore, we propose a hierarchical semantic segmentation framework incorporating an attribute prompt tuning method. This approach is designed to enhance the model's adaptation to the attribute segmentation task and effectively leverage attribute features to produce better semantic segmentation results. Correspondingly, we construct a Visual Hierarchical Semantic Classes (VHSC) benchmark, meticulously annotating shared-attributes at the pixel level to conduct the experiments. Extensive experiments on the VHSC benchmark showcase the superior performance of our method compared to existing zero-shot semantic segmentation methods, achieving mIoU of 73.0% and FBIoU of 87.5%. The VHSC benchmark and our code will be released to the community.

Affinity3D: Propagating Instance-Level Semantic Affinity for Zero-Shot Point Cloud Semantic Segmentation

Pass3d: Precise And Accelerated Semantic Segmentation For 3d Point Cloud

Learning Hybrid Semantic Affinity for Point Cloud Segmentation

Associate Semantic-Instance Segmentation of 3D Point Clouds Based on Local Feature Extraction

SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data

Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation

Transferring CLIP's Knowledge into Zero-Shot Point Cloud Semantic Segmentation

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Associatively Segmenting Instances and Semantics in Point Clouds

Zero-Shot Dual-Path Integration Framework for Open-Vocabulary 3D Instance Segmentation

Multi-to-Single Knowledge Distillation for Point Cloud Semantic Segmentation

Semantic Labeling and Instance Segmentation of 3D Point Clouds Using Patch Context Analysis and Multiscale Processing

Semantic Segmentation of Point Cloud Scene via Multi-Scale Feature Aggregation and Adaptive Fusion

Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation

Advancing zero-shot semantic segmentation through attribute correlations

Rethinking Few-shot 3D Point Cloud Semantic Segmentation

Multimodality Helps Few-Shot 3D Point Cloud Semantic Segmentation

Unsupervised Domain Adaptive Point Cloud Semantic Segmentation.

Few-shot 3D Point Cloud Semantic Segmentation